Is it possible to create a Consul Cluster across the Internet with agents on different LANs?

I want to create a Consul cluster with some Raspberry Pi computers and a central server on the Internet. Each of the RPi computers will be on a different LAN.

NOTE: I appreciate that having a single Consul server is not suitable for production, but this is an idea I am playing with.

I have configured the Consul server on the Internet and have enabled ACLs for both of the software. I have installed Consul on the RPi computers and indeed they can communicate with the central server, but the health check is flip-flopping from healthy to not-healthy.

Looking into this I think it is because the address that the client has advertised the LAN address to the server rather than its WAN address. I have tried to use the advertise_addr_wan and translate_wan_addrs options but this has not helped.

I have been looking at the architecture diagrams for Consul and see that it is possible to connect servers across the WAN from different data-centres but not (obviously) clients. Do I need to run a server and a client process on each of the RPi so that each one is its own Data Centre and then connect the DC servers together across the WAN?

After I have set this up I want to configure Nomad to use Consul and have a Nomad agent on each of the client RPi computers. I assume that I will encounter similar issues with IP addressing. Am I correct?

I do not want to resort to having to run a VPN for all the traffic between the nodes nor do I want to install Docker and run Kubernetes because the app I will be deploying is not natively in a container.

Any ideas on how to accomplish this is very gratefully received.

Hi @russellseymour,

Consul client agents are designed to be deployed within a single LAN with latency of 10ms or less. See this thread for a similar discussion.

Hope this helps.