The consul client connect to consul server in another network

[zhangzerui20]

As shown in the picture above, I am dealing with a complex network scenario.

There are two VPCs, the network between VPCs cannot be connected.

There are three consul servers deployed in VPCB, and their IP is vpcB.0, vpcB.1, vpcB.2.

Cloud vendors provide a network connection scheme between two VPCs. After configuration, the data packets sent to vpcA.0 in the VPCA will be forwarded to vpcB.0, and the data packets sent to vpcA.1 will be sent to vpcB.1. The data packet sent to vpcA.2 will be sent to vpcB.2.

In this kind of network, I want to run the consul client in VPCA and connect to the consul server in VPCB. But it doesn’t work properly. I tried the following two configurations:

1. Set the -advertise of consul server to vpcB.0,vpcB.1,vpcB.2 , set the -retry-join of consul client to vpcA.0, vpcA.1, vpcA.2 . In this configuration, the consul server cluster can operate normally. However, the consul client will find vpcB.0, vpcB.1, vpcB.2 through gossip, because the consul client in VPCA cannot directly access these ips, the consul client will report an error timeout.
2. Set the -advertise of consul server to vpcA.0,vpcA.1,vpcA.2. In this configuration, the consul server cannot work normally because the consul server in the VPCB cannot access the address in the VPCA.

I spent a lot of time searching for documents and issues, but I couldn’t find a solution. Whether there is any configuration here, it can be returned to different addresses of the client and server. Or is there a problem with the way I use it, in my case, is there any solution?

[mkeeler]

All consul agents advertised addresses within a single consul datacenter must be reachable using that address from all nodes in the datacenter. That goes for communications between servers, clients, or a client and a server.

With your networking setup it doesn’t look possible to satisfy those requirements. Like you have figured out, setting the advertise address to one IP or the other causes different behavior but neither results in a fully working consul cluster.

[zhangzerui20]

If the clients can talk to the servers and the servers can talk to the servers, but the servers cannot talk to the clients, what problems will this situation cause?

[blake]

For anyone who might come across this thread, this question was asked and answered in another post.

Consul server can not talks to clients

The health checks for the client nodes will be intermittently unhealthy. This will affect service discovery both via the HTTP API and DNS. If utilizing service mesh it will also potentially alter things to steer traffic away from those nodes until they are healthy again. You may be able to see this in the consul UI with the nodes flapping between healthy and unhealthy status depending on how long the period of unhealthiness lasts.