How do I point a domain name to a Nomad job running in a cluster?

Nomad
1

I’ve got 3 machines running a cluster of consul/nomad agents.

I deploy an nginx job and I can hit the IP address/port of the allocation and view the files served by the nginx job.

How do I connect my domain name to that job? My domain is pointing to DigitalOcean’s nameservers where I can create DNS records. I could manually create an A record from my domain to the IP address of the machine with the allocation running the nginx job. But what if the job moves to a different machine?

I know that from a machine within the cluster I can dig @127.0.0.1 -p 8600 some-name.service.consul ANY to get the IP of the agent running the service. But how do I hit that from an external domain?

I know this isn’t an unusual ask. I just can’t seem to find any material describing this (or the material that does describe it is so foreign that I don’t realize that it’s what I’m looking for).

2

Take a look at fabio

3

The Hashi docs related to Nomad and Fabio still end with “Configure your AWS load balancer to point to the machines running your Fabio jobs.” https://learn.hashicorp.com/tutorials/nomad/load-balancing-fabio#step-5-place-nomad-client-nodes-behind-aws-load-balancer

That’s the step I want to handle. What’s the best way to manually manage a HA loadbalancer.

Right now, I have 3 “load-balancer” machines that each run a system job of NGINX to load-balance to N other jobs running an NGINX web server. I have 3 A records in my DNS connecting my domain name to those 3 load balancers. I suppose this is basically what a managed loadbalancer like AWS’s version does behind the scenes. I’d love to know what else there is.

4

Fabio needs the key for the app routes. Try tagging your nomad service with urlprefix-myapp.mydomain.tld.

myapp.mydomain.tld => nginx-lb => fabio:9999 => webserver:port
5

@eihli I think you’ve got the right approach here if you don’t want to use a managed load balancer. To summarize:

  • point the CNAME to the IP of each machine that you want to be part of the load balancer.
  • have a system job on each of those machines that listens on a static port and routes traffic to backends (this could be Nginx, HAProxy, Fabio, whatever)

If you intend to rotate out those machines, you’ll need to either update the CNAME (use a short TTL!) or have the machines use a floating virtual IP so that you can move the IP address to a different machine and not have to wait for DNS TTLs.

And you’re right that effectively this is what a managed load balancer is doing. The added bit that makes that seem a little more “magic” is they add another CNAME in-between, so that ends up like this:

  • point your CNAME to their CNAME
  • their CNAME points to the IPs of each machine in the load balancer
  • their load balancer software (typically HAProxy or some custom Erlang thing, depending on which provider) listens on a static port and routes traffic to your backends.
1 Like