Use Vagrant to stand up multiple VMs with Bridged IP

Posted by paul on 2017.03.12

Tried to learn Ansible, but I learned Vagrant first

Recently I started going through "Ansible for DevOps" to learn more about Ansible. It's an excellent book as I learned more about Ansible by just following the book versus when I tried following online tutorials. But I got to the Ansible part only after a lengthy detour to learn Vagrant. The book begins with how to set up Vagrant. Using Vagrant while practicing Ansible or any other configuration management tool is absolutely necessary as you will need to stand up new OS instances quite often.

The book assumes you are using a Vagrant image uploaded by the author. I wanted to use the official CentOS 7 image from CentOS. I also wanted to use Ansible controller I had set up on my Mac previously, without any modification. This combination prevented me from following the tutorial successfully. After some googling/hacking, I got Vagrant and Ansible controller on my Mac working together and was able to proceed with the rest of the book. I did not find tutorials online that seemed to address my situation exactly so I am sharing it here.

This tutorial will work best if you are in a home or office LAN with spare, unused IPs. If you are trying this in LAN of your work, make sure to ask your network administrator if you can use a few of the IPs. This set up will not work as well if you are going to different, public LANs, like at Starbucks. My tutorial is done on a Macbook laptop running 10.10.

Summary of steps

  1. Install VirtualBox.
  2. Install Vagrant.
  3. Set up a Vagrant CentOS 7 VM, with internal IP only.
  4. Set up a Vagrant CentOS 7 VM, with a Bridged IP.
  5. Set up 2 Vagrant VMs, each with a unique, Bridged IP.

Install VirtualBox

Install VirtualBox on your Mac. This tutorial does not require "VirtualBox Extension Pack".

Install Vagrant

Install Vagrant on your Mac by following https://www.vagrantup.com/docs/installation. Installing it is simple.

Vagrant has one useful plugin that you should install. This plugin automatically updates /private/etc/hosts file on your Mac whenever a Vagrant VM is added or removed. Very handy plugin. You can install the plugin with following command in Terminal. This plugin will come into use when you start naming the Vagrant VMs and not use the default name "default".

vagrant plugin install vagrant-hostsupdater

Set up a Vagrant CentOS 7 VM, with internal IP only.

We will start with building a plain Vagrant VM with no customization. Each Vagrant environment needs own folder. And in each folder is a text file, Vagrantfile. This file controls the Vagrant environment. By editing this file, you add more VMs, assign static IPs, configure amount of RAM for the VM, etc etc.

Create a folder to store Vagrant config files. You can check these files into Git repository. I keep all my git repo files in ~/codes/. So in this tutorial, all Vagrant files are stored with following folder structure on my Mac.

~/codes/vagrants/centos7-initial/
~/codes/vagrants/centos7-bridged/

In ~/codes/vagrants/, create folder centos7-initial for the first CentOS 7 image. Change into directory, centos7-initial. Run following command, which creates file, Vagrantfile, in your present working directory (centos7-initial).

vagrant init centos/7

There is now a text file, ~/codes/vagrants/centos7-initial/Vagrantfile

Maclaptop:centos7-initial paul$ ls
Vagrantfile
Maclaptop:centos7-initial paul$

Now fire up the CentOS 7 Vagrant VM. Because your Mac does not have the CentOS 7 Vagrant box image file yet, Vagrant will download the image file for you. But once it's downloaded, you don't need to download it again. The file is about 400M in size.

vagrant up

Here are all the commands in one example.

Maclaptop:~ pchu$
Maclaptop:~ pchu$ cd ~/codes/vagrants/

Maclaptop:vagrants pchu$ mkdir centos7-initial

Maclaptop:vagrants pchu$ cd centos7-initial

Maclaptop:centos7-initial paul$ vagrant init centos/7
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

Maclaptop:centos6-initial paul$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'centos/7' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'centos/6'
    default: URL: https://atlas.hashicorp.com/centos/6
==> default: Adding box 'centos/7' (v1611.01) for provider: virtualbox
    default: Downloading: https://atlas.hashicorp.com/centos/boxes/7/versions/1611.01/providers/virtualbox.box
    default: Progress: 3% (Rate: 2160k/s, Estimated time remaining: 0:03:12)
...
...

You will see many lines of text scroll by. If you see following, your Vagrant VM with CentOS 7 is ready.

...
==> default: Machine booted and ready!
...

Now you can ssh into the VM. While still in folder centos7-initial, run vagrant ssh to ssh into the new VM, and become root. Below is what you should see. In line 1, you are sshing into the CentOS 7 VM as user "vagrant" on the VM. The user "vagrant" was already there when the VM was downloaded. Lines 1 - 2 are when you are logged into the Mac. Lines 3 - 5 are when you are logged into the CentOS 7 VM. Line 5 is where you are the root of the CentOS 7 VM. Pay attention to the shell prompt when working with Vagrant so that you are always aware what OS you are in, your Macbook or the Vagrant VM. If you are not caureful, you could accidentally delete files on your Mac when you actually meant to run the command on your VM.

Maclaptop:centos7-initial paul$
Maclaptop:centos7-initial paul$ vagrant ssh
[[email protected] ~]$
[[email protected] ~]$ sudo su -
[[email protected] ~]#

Check the IP of the VM, and you will see it's something like 10.0.2.xxx. Your VM is running behind NAT. This means you can ssh into the VM only from your Mac via Vagrant. You cannot ssh into the VM from a second Terminal window. You cannot scp files from you Mac to the VM either. In the example below, the IP is 10.0.2.15. The VM at this stage is not really useful for anything.

[[email protected] ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:a1:66:8e brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0
inet6 fe80::5054:ff:fea1:668e/64 scope link
valid_lft forever preferred_lft forever
[[email protected] ~]#

Let's create a new Vagrant VM with a Bridged IP. Before we leave folder ~/codes/vagrant/centos7-initial/, we need to delete the VM you just created.

In the CentOS 7 VM, log out from root to user 'vagrant'. And log out from user vagrant in the Vagrant VM. Now you are in your Mac's terminal, and back in this folder ~/codes/vagrants/centos7-initial/. Delete the Vagrant VM by running following command, which you run within the macOS. Using -f allows you to delete the VM without having to answer a prompt asking for confirmation.

Maclaptop:centos7-initial paul$ vagrant destroy -f

Set up a Vagrant CentOS 7 VM, with a Bridged IP

To set up a Vagrant VM to act truly like a server (at least inside of your LAN), you need to give it a Bridged IP. Vagrant documentation calls it a public IP instead of Bridged IP. And even though it's called a public IP in this tutorial, in reality it is an IP on your LAN (so not really public). My LAN uses 192.168.11.xxx. So I will assign 192.168.11.201 to my Vagrant VM. This allows me to interact with the Vagrant VM as if it were a real server. I can ssh into it, copy files to it using scp, and browse website running on it.

To do this, you will create new directory: ~/codes/vagrants/centos7-bridgeip/

mkdir ~/codes/vagrants/centos7-bridgeip

Next you will edit ~/codes/vagrants/centos7-bridgeip/Vagrantfile. Delete everything in file Vagrantfile and paste in following text into the file.


Vagrant.configure("2") do |config|
  config.vm.box = "centos/7"
  config.vm.network "public_network", ip: "192.168.11.201", bridge: "en0: Wi-Fi (Airport)", bootproto: "static", gateway: "192.168.11.1"
end

Some of the values in the example will need to be changed. You will need to change value of ip and value of gateway to match your environment. If your Mac's IP is 192.168.11.5, it's probably safe to use 192.168.11.201 as the IP and use 192.168.11.1 as gateway.

Value of bridge may require some extra work. I use Wi-Fi on Macbook exclusively, so if you also use Wi-Fi on your Mac, you can use bridge: "en0: Wi-Fi (Airport)". But if you use Ethernet, you will need to figure out what to use.

Optional: how to figure out what network name Vagrant can work with

Here's how to get the network device name that Vagrant can work with.

  • While in ~/codes/vagrants/centos7-bridgeip/, edit Vagrantfile. Change value of bridge to something like "en0: Wi-Fi" (which is intentionally set wrong) and save Vagrantfile.
  • Run "vagrant up" to start the VM.
  • After a few lines of text, Vagrant will pause and show this line
  • Specific bridge 'en0: Wi-Fi' not found. You may be asked to specify
  • You will see a list of possible choices. On my early 2013 Macbook Pro, I used an Thunderbolt Ethernet adapter to connect with ethernet cable and disabled Wi-Fi. I got following as first choice:
  • en4: Thunderbolt Ethernet
  • I copied that text into clipboard.
  • Press ctrl-c to cancel "vagrant up" command. You may need to issue ctrl-c 2 or 3 times.
  • Open Vagrantfile to edit.
  • Change value of bridge: as shown below.
    Vagrant.configure("2") do |config|
    config.vm.box = "centos/7"
    config.vm.network "public_network", ip: "192.168.11.201", bridge: "en4: Thunderbolt Ethernet", bootproto: "static", gateway: "192.168.11.1"
    end
  • End of discovering network device name to use with Vagrant

Resuming: Set up a Vagrant CentOS 7 VM, with a Bridged IP

  1. You have a valid value to use with bridge:, either bridge: "en0: Wi-Fi (Airport)" or bridge: "en4: Thunderbolt Ethernet" or etc.
  2. Once Vagrantfile is updated with those 3 values, you are ready to fire up the new VM.
  3. Bring up the VM
  4. vagrant up
  5. Wait for the VM to start.
  6. Once it is up, run "vagrant ssh" to ssh in as user "vagrant".
  7. Once logged into the CentOS 7 VM, run "sudo su -"
  8. Check IP address
  9. ip a
  10. You will notice eth1 is there, but no IP is actually active.
  11. [[email protected] ~]# ip a
        1: lo:  mtu 65536 qdisc noqueue state UNKNOWN qlen 1
            link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
            inet 127.0.0.1/8 scope host lo
               valid_lft forever preferred_lft forever
            inet6 ::1/128 scope host
               valid_lft forever preferred_lft forever
        2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
            link/ether 52:54:00:22:5b:53 brd ff:ff:ff:ff:ff:ff
            inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
               valid_lft 81084sec preferred_lft 81084sec
            inet6 fe80::5054:ff:fe22:5b53/64 scope link
               valid_lft forever preferred_lft forever
        3: eth1:  mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
            link/ether 08:00:27:a6:5c:5a brd ff:ff:ff:ff:ff:ff
        [[email protected] ~]#
                
  12. Eth1 is the 2nd NIC we added, which is also known as public IP (or bridged IP). This eth1 is not activate yet and you need bring it up with "ifup eth1" or reboot the VM. It should come up with the IP we assigned. When you run "ip a" again, you should see eth1 has IP address 192.168.11.201, which was specified in Vagrantfile.
  13.     [[email protected] ~]# ifup eth1
        [[email protected] ~]# ip a
        1: lo:  mtu 65536 qdisc noqueue state UNKNOWN qlen 1
            link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
            inet 127.0.0.1/8 scope host lo
               valid_lft forever preferred_lft forever
            inet6 ::1/128 scope host
               valid_lft forever preferred_lft forever
        2: eth0:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
            link/ether 52:54:00:22:5b:53 brd ff:ff:ff:ff:ff:ff
            inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
               valid_lft 81051sec preferred_lft 81051sec
            inet6 fe80::5054:ff:fe22:5b53/64 scope link
               valid_lft forever preferred_lft forever
        3: eth1:  mtu 1500 qdisc pfifo_fast state UP qlen 1000
            link/ether 08:00:27:a6:5c:5a brd ff:ff:ff:ff:ff:ff
            inet 192.168.11.201/24 brd 192.168.11.255 scope global eth1
               valid_lft forever preferred_lft forever
            inet6 fe80::a00:27ff:fea6:5c5a/64 scope link
               valid_lft forever preferred_lft forever
        [[email protected] ~]#
        
  14. Open a new Terminal window on your Mac and try pinging 192.168.11.201. You should be able to ping successfully. Note you cannot ssh into the CentOS 7 VM at 192.168.11.201 yet.
  15. Let's complete 3 little tasks within the VM with the Bridged IP so you can ssh into it from a 2nd Terminal window. This will allow you to ssh into the VM at 192.168.11.201 from a 2nd Terminal window or even another computer on your LAN.
  16. Back at your 1st Terminal window where you are logged into the CentOS 7 VM, add a user (I use paul here) and install 'vim' text editor. I'm using yum command to check it can pull files from the internet. This completes 1st and 2nd tasks.
  17. Here are the commmands.
  18. [[email protected] ~]# useradd paul
    [[email protected] ~]# passwd paul
    [[email protected] ~]# Changing password for user paul.
    [[email protected] ~]# New password:
    
    [[email protected] ~]# yum install -y vim
  19. Third task is updating /etc/ssh/sshd_config within the CentOS 7 VM.
    • Open /etc/ssh/sshd_config to edit.
    • Change following line. It's probably line 79.
    
        # Line to update BEFORE
        PasswordAuthentication no
    
        # Line to update AFTER
        PasswordAuthentication yes
  20. Save /etc/ssh/sshd_config and restart sshd service of the CentOS 7 VM.
  21.     systemctl restart sshd
  22. Now, you can ssh into the CentOS 7 VM from the 2nd Terminal window or any other computer on your LAN via IP 172.168.11.201.

You now have a virtual CentOS 7 server on your LAN that can do everything a real server in your LAN can do, including sshing in, copying files in, and host a test website on it for dev work. Now that you are done with this VM, log out of the VM (issue "exit" twice). Back in the Mac's Terminal, destroy the VM with "vagrant destroy -f".

Set up 2 Vagrant VMs, each with a unique, Bridged IP

Now that we know how to create a VM with Bridged IP, let's create 2 identical VMs. With a properly configured Vagrantfile, you can create 2 or more VMs by simply issuing "vagrant up" once. This is extremely useful when practicing with Ansible as it will allow you to stand up many VMs quickly.

In my example with the 2nd VM (which used Bridged IP), I used 192.168.11.201. In the next 2 VMs, I will use 192.168.11.202 and 192.168.11.203. You could reuse same IPs when following this tutorial, but you would need to delete a line in ~/.ssh/known_hosts on your Mac every time a new VM is brought. To avoid this hassle, I will use different IP for each VM.

Make sure the VM with 192.168.11.201 is destroyed. Not just shutdown but destroyed (aka deleted). I usually destroy VM after using it to keep my system clean.

Copy folder ~/codes/vagrants/centos7-bridgeip
to ~/codes/vagrants/centos7-bridgeip-qt-2. When I practice modifying Vagrantfile, I simply copy from a working Vagrant directory. That way, when something doesn't work, you can go back to a working copy..

Change directory to ~/codes/vagrants/centos7-bridgeip-qt-2

While in directory ~/codes/vagrants/centos7-bridgeip-qt-2/, open Vagrantfile to edit. Delete everything in Vagrantfile, and paste in following. We are configuring both VMs to get 512MB of RAM each.

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # General Vagrant VM configuration.
  config.vm.box = "centos/7"
  config.ssh.insert_key = true
  config.vm.synced_folder ".", "/vagrant", disabled: true
  config.vm.provider :virtualbox do |v|
    v.memory = 512
    v.linked_clone = true
  end

  # Server 1.
  config.vm.define "cv01.lan.vag" do |web|
    web.vm.hostname = "cv01.lan.vag"
    web.vm.network :public_network, ip: "192.168.11.202", bridge: "en0: Wi-Fi (Airport)", bootproto: "static", gateway: "192.168.11.1"
  end

  # Server 2.
  config.vm.define "cv02.lan.vag" do |web|
    web.vm.hostname = "cv02.lan.vag"
    web.vm.network :public_network, ip: "192.168.11.203", bridge: "en0: Wi-Fi (Airport)", bootproto: "static", gateway: "192.168.11.1"
  end
end

There are some changes with this new Vagrantfile.

  1. # General Vagrant VM configuration: This sets same config for all VMs.
  2. # Server 1: This sets settings for server 1 at 192.168.11.202.
  3. # Server 2: This sets settings for server 2 at 192.168.11.203.

Obviously you need to give a unique IP and name to each VM.

Make sure you are in directory ~/codes/vagrants/centos7-bridgeip-qt-2/, and run "vagrant up"

As the VMs are brought online, you will be given this prompt below and asked to enter a password. The password Vagrant is looking for is the root password of YOUR Mac. It is trying to add a few lines of text to file /private/etc/hosts on your Mac which can be edited only with root privilege. You need to enter the Mac root password only once as long as you don't reboot. You get this prompt here because we installed vagrant plugin "vagrant-hostsupdater" earlier. With this done on your Mac, you can reach your VMs using hostname, in this case cv01.lan.vag and cv02.lan.vag.

==> cv01: [vagrant-hostsupdater] This operation requires administrative access. You may skip it by manually adding equivalent entries to the hosts file.
Password:

Later when you check /private/etc/hosts on your Mac, you will notice the lines added at the end of the file

Once the 2 VMs are online, try running "vagrant ssh". In earlier example with only 1 VM online, you sshed into the VM with "vagrant ssh". But with 2 VMs, you have to specify which VM to use. So you would run "vagrant status" to see list of VMs running, and then run "vagrant ssh _name_" to ssh into the VM.

PCs-MacBook-Pro:centos7-bridgedip-qt2 paul$
PCs-MacBook-Pro:centos7-bridgedip-qt2 paul$ vagrant ssh
This command requires a specific VM name to target in a multi-VM environment.
PCs-MacBook-Pro:centos7-bridgedip-qt2 paul$ vagrant status
Current machine states:

cv01.lan.vag                      running (virtualbox)
cv02.lan.vag                      running (virtualbox)

This environment represents multiple VMs. The VMs are all listed
above with their current state. For more information about a specific
VM, run `vagrant status NAME`.
PCs-MacBook-Pro:centos7-bridgedip-qt2 paul$ vagrant ssh cv01
[[email protected] ~]$

You should be able to ping both cv01.lan.vag and cv02.lan.vag due to updated added to /private/etc/hosts on your Mac. And if you updated both servers by adding a user, modify /etc/ssh/sshd_config, and restart sshd service, you can ssh into both servers. However that would be a lot of typing if you had to do that on 10 VMs. And this is where "Shell Provisioner" and Ansible come in.

In my next blog, I will explain how to configure a CentOS 7 to be an Ansible client. And then I will explain how to use "Shell Provisioner" in Vagrant to add Ansible client to all Vagrant VMs without any manual work. In the end, you will be able to use "vagrant up" once and bring online multiple VMs already configured with Ansible client.