We are running a Nomad server cluster on AWS and plan to potentially have thousands of Nomad clients connecting to it from different Cloud Providers (not AWS/GCP/Azure) and private Data Centers.
Exposing servers directly to the public internet is never a good practice so we tried to put the cluster (specifically port 4647) behind a Network Load Balancer and pass the NLB URL to the clients under servers configuration option
servers = [ "some.nlb.acme.com" ]
This NLB URL is also the rpc advertise address of the Nomad servers.
With this configuration, it seems like the Nomad client tries to get all 3 expected servers IP from the same URL and ends up having the same IP 3 times -
root@nomad-client:~# nomad node config -servers 188.8.131.52:4647 184.108.40.206:4647 220.127.116.11:4647
(This IP is the one of the NLB interfaces IP in AWS and not a Nomad server IP)
This seems like a bad way to achieve a secured and working setup.
Is there another method / case study / recommendation for setting up a Nomad cluster in a secure way when the clients are not in the same VPC / cloud provider?