Local Nomad Cluster Using Vagrant

Nomad
9

Hi @egmanoj,

Thanks for using Nomad. I haven’t set things up with Ansible, but the following process works for me reliably using Vagrant and Nomad. I hope it is useful for you.

Note: I use the Vagrantfile in the Nomad repo

Host machine

  • Install Vagrant
  • Install Virtualbox
  • mkdir -p /opt/gopath/src/github.com/hashicorp
  • sudo chown <username> /opt/gopath/src
  • cd /opt/gopath/src/github.com/hashicorp
  • git clone https://github.com/hashicorp/nomad
  • cd nomad
  • vagrant up linux nomad-server01 nomad-client01
  • Get coffee because that’s gonna take a bit
  • vagrant status

You should see a list of a boxes like this

linux                     not created (virtualbox)
linux-ui                  not created (virtualbox)
freebsd                   not created (virtualbox)
nomad-server01            not created (virtualbox)
nomad-client01            not created (virtualbox)
nomad-server02            not created (virtualbox)
nomad-client02            not created (virtualbox)
nomad-server03            not created (virtualbox)
nomad-client03            not created (virtualbox)

Single Server Setup

nomad-server01

  • vagrant ssh nomad-server01
  • make bootstrap - installs dependencies
  • make dev - builds Nomad from source in the current directory bin folder
  • cp bin/nomad /opt/gopath/bin/nomad
  • sudo mkdir /etc/nomad.d/data
  • sudo chown vagrant /etc/nomad.d
  • vim /etc/nomad.d/server.hcl
log_level= "DEBUG"
datacenter = "dc1"
data_dir = "/etc/nomad.d/data"
enable_debug = true

server {
  enabled = true
  bootstrap_expect = 1 # this could also be 3 or 5 if you need/want to test with multiple servers
}

addresses {
  rpc = "192.168.56.11"
  serf = "192.168.56.11"
}
  • sudo vim /etc/systemd/system/nomad.service
[Unit]
Description=Nomad Service
Requires=network-online.target
After=network-online.target

[Service]
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity
User=root
EnvironmentFile=-/etc/sysconfig/nomad
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/opt/gopath/bin/nomad agent $OPTIONS -config=/etc/nomad.d
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
StandardOutput=journal

[Install]
WantedBy=multi-user.target
  • sudo systemctl enable nomad

Client Setup

  • vagrant ssh nomad-client01
  • cp bin/nomad /opt/gopath/bin/nomad
  • sudo mkdir /etc/nomad.d/data
  • sudo chown vagrant /etc/nomad.d
  • vim /etc/nomad.d/client.hcl
log_level= "DEBUG"
datacenter = "dc1"
data_dir = "/etc/nomad.d/data"
enable_debug = true

client {
  enabled = true
  server_join {
    retry_join = ["192.168.56.11"]
  }
}
  • sudo vim /etc/systemd/system/nomad.service
[Unit]
Description=Nomad Service
Requires=network-online.target
After=network-online.target

[Service]
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity
User=root
EnvironmentFile=-/etc/sysconfig/nomad
Environment=GOMAXPROCS=2
Restart=on-failure
ExecStart=/opt/gopath/bin/nomad agent $OPTIONS -config=/etc/nomad.d
ExecReload=/bin/kill -HUP $MAINPID
KillSignal=SIGINT
StandardOutput=journal

[Install]
WantedBy=multi-user.target
  • sudo systemctl enable nomad

At this point I have a running cluster with one server and one client. I can repeat the process and adjust the configuration to add more servers (bootstrap_expect=3 for example) or clients. Don’t forget to use an odd number of servers (1, 3, or 5) so that you can calculate quorum effectively. I recommend you start with the simplest possible config (single server) and progressively expand.

Please let me know if that helps, or if you run into issues. I’m currently working on making this easier for users and would welcome any feedback.

1 Like