Joining client to Consul from different VPC on DigitalOcean

I have 3 consul servers on DigitalOcean with the following config:

datacenter     = "dc1"
data_dir       = "/opt/consul"
bind_addr      = ""
advertise_addr = "10.114.0.x" --> x = 1, 2, 3 respectively
client_addr    = ""
retry_join     = ["", "", ""]

connect {
  enabled = true
ui               = true
server           = true
bootstrap_expect = 3

All 3 servers have 3 IPs: Private, Public, Floating

This is fine and I have a few clients that connect to it in the same VPC with configuration like this:

datacenter     = "dc1"
data_dir       = "/opt/consul"
advertise_addr = ""
retry_join     = ["", "", ""]

Now, a new client comes and he has a different account/team with its own separate billing settings. This means that he has his own VPC for his account.

If I create a client for him with the client configuration above, it fails with the following:

[ERROR] agent.client: RPC failed to server: method=Catalog.ListDatacenters server= error="rpc erro r getting client: failed to get conn: dial tcp <nil>-> connect: no route to host"

[ERROR] client: error discovering nomad servers: error="client.consul: unable to query Consul datacenters: Unexpected response code: 500 (rpc error getting client: failed to get conn: dial tcp <nil>-> connect: no route to host)"

It also logs this for the other 2 servers. This is not a big surprise, since they’re on a different VPC.

I tried many sensible permutations of the following:

  • Use public IPs for retry_join
  • Use floating IPs for retry_join
  • Use public IP for bind_addr
  • Use public IP for advertise_addr

By sensible, I mean setting the 2 wan attributes together, etc.

One approach could be to create another datacenter for the client account and join the 2 with consul join -wan but that’s not what I want to do.

Is there an option to join a client outside of the VPC to a consul mesh without creating another datacenter?