Local development workflow with Nomad, Consul and Docker

Nomad
1

I’m having a bad time testing out my Nomad cluster locally on my laptop running OSX. I first ran it inside Docker (which is not recommended) but that broke things like memory detection and it didn’t like template-stanzas.

I then decided to run Nomad and Consul outside of Docker using bridge networking (the default), the containers live in the default 172.17.0.0/16 address space, but when looking up services in Consul, the containers are receiving 127.0.0.1, which won’t work inside the containers, they need the bridged IP.

Can anyone tell me how they run their local single node clusters for testing purposes and give me any hints?

It seems like Weave had the right idea for local development, but I have hard time finding any recent guides on how to run it with Nomad, and since version 2 they seem to focus on Docker Swarm.

1 Like
2

I’ve stumbled upon two different (officially looking) Vagrant guides:

https://github.com/hashicorp/nomad/tree/master/demo/vagrant (recent)

https://github.com/hashicorp/nomad-guides/tree/master/operations/provision-nomad/dev/vagrant-local (very old)

I’m giving them a spin now.

1 Like
3

Found a third one here: https://github.com/hashicorp/nomad-guides/tree/master/provision/vagrant

1 Like
4

Hey!

Did you found something yet? I’m actually dealing with the same issue and can’t work it out :confused:

1 Like
5

Hi @SantoDE, sorry that you’re struggling with this as well, I really with the development workflow would be better. I am still unable to get proper template-stanzas to work with Docker on my Macbook.

I didn’t find anything yet, but I see a few options:

  • Try to patch one of the available Vagrant options, and try to make it fit my needs of a single-node cluster running Nomad (server+client), Consul and Dnsmasq. But I’m not that much of a fan of the Vagrant workflow, maybe a Packer builder to VMWare or Virtualbox would be better.
  • Maybe just setup a spare laptop to run everything.
  • Testing in production :sob:
  • Waiting for a Hashicorp product manager to feel bad for us and find a solution.
1 Like
6

Hi @kaspergrubbe! I’m an engineer on the Nomad team who uses macOS as my daily driver, so I probably have some insights here. My typical workflow for single-node development is to use the Vagrantfile at the root of the https://github.com/hashicorp/nomad/blob/master/Vagrantfile and spin up the “linux-ui” box defined there (because I like to look at the Nomad web UI sometimes).

With the configurations below I can access Nomad and Consul both on localhost from the macOS host.

I use the following Consul config:

{
   "ui": true,
  "bootstrap_expect": 1,
  "server": true,
  "log_level": "INFO",
  "data_dir": "/var/consul/data",
  "bind_addr": "{{ GetInterfaceIP \"eth0\" }}",
  "client_addr": "0.0.0.0",
  "advertise_addr": "{{ GetInterfaceIP \"eth0\" }}",
  "connect": {
    "enabled": true
  },
  "ports": {
    "grpc": 8502
  }
}

And the following Nomad config:

log_level = "DEBUG"
data_dir = "/var/nomad"
name = "testing"
enable_debug = true

client {
  enabled = true
}

plugin "raw_exec" {
  config {
    enabled = true
  }
}

server {
  enabled          = true
  bootstrap_expect = 1
}
1 Like