I’ve got 3 machines running a cluster of consul/nomad agents.
I deploy an nginx job and I can hit the IP address/port of the allocation and view the files served by the nginx job.
How do I connect my domain name to that job? My domain is pointing to DigitalOcean’s nameservers where I can create DNS records. I could manually create an A record from my domain to the IP address of the machine with the allocation running the nginx job. But what if the job moves to a different machine?
I know that from a machine within the cluster I can
dig @127.0.0.1 -p 8600 some-name.service.consul ANY to get the IP of the agent running the service. But how do I hit that from an external domain?
I know this isn’t an unusual ask. I just can’t seem to find any material describing this (or the material that does describe it is so foreign that I don’t realize that it’s what I’m looking for).
The Hashi docs related to Nomad and Fabio still end with “Configure your AWS load balancer to point to the machines running your Fabio jobs.” https://learn.hashicorp.com/tutorials/nomad/load-balancing-fabio#step-5-place-nomad-client-nodes-behind-aws-load-balancer
That’s the step I want to handle. What’s the best way to manually manage a HA loadbalancer.
Right now, I have 3 “load-balancer” machines that each run a system job of NGINX to load-balance to N other jobs running an NGINX web server. I have 3 A records in my DNS connecting my domain name to those 3 load balancers. I suppose this is basically what a managed loadbalancer like AWS’s version does behind the scenes. I’d love to know what else there is.
Fabio needs the key for the app routes. Try tagging your nomad service with
myapp.mydomain.tld => nginx-lb => fabio:9999 => webserver:port
@eihli I think you’ve got the right approach here if you don’t want to use a managed load balancer. To summarize:
- point the CNAME to the IP of each machine that you want to be part of the load balancer.
- have a system job on each of those machines that listens on a static port and routes traffic to backends (this could be Nginx, HAProxy, Fabio, whatever)
If you intend to rotate out those machines, you’ll need to either update the CNAME (use a short TTL!) or have the machines use a floating virtual IP so that you can move the IP address to a different machine and not have to wait for DNS TTLs.
And you’re right that effectively this is what a managed load balancer is doing. The added bit that makes that seem a little more “magic” is they add another CNAME in-between, so that ends up like this:
- point your CNAME to their CNAME
- their CNAME points to the IPs of each machine in the load balancer
- their load balancer software (typically HAProxy or some custom Erlang thing, depending on which provider) listens on a static port and routes traffic to your backends.