So, I got a Nomad/Vault/Consul cluster up and going using this excellent resource: nomad/terraform/aws at main · hashicorp/nomad · GitHub
Everything was working, but I wanted to upgrade to Ubuntu 20.04, since 16.04 was now EOL. I’m having trouble getting DNS lookup to work in the Docker containers. In other words, lookup of, say, “postgres.service.consul” isn’t working.
- I got rid of systemd-resolved on these systems, and consul is listening on on port 53 (on all machines, both nomad servers and clients)
- Consul is started with: /usr/local/bin/consul agent -config-dir=/etc/consul.d -dns-port=53 -recursor=172.31.0.2
On the Nomad client, a ‘dig postres.service.consul’ will return the IP of the container. But inside the container, the lookup fails. However, inside the container I can look up non-cluster address, e.g. google.com
/etc/resolv.conf on the host:
/etc/resolv.conf in the container:
But if I change it to 172.31.0.1 inside the container, it still will not lookup, and in fact times out.
- I’m sure I’m missing something simple, but I don’t know what.
- Any other troubleshooting information I can dig up?