Best HA with 3 nodes (same datacenter or not)

I am using consul as service discovery, vault backend, and with patroni (Postgres) but currently, all consul nodes are in the same data center/region, therefore I was wondering what are the pros/cons of having each node in a different data center/location, will latency be an issue? or it is probably better to have 2 extra (a total of 5) but these new two nodes in different locations each.

Not a big deal, but Consul is the backend, Vault uses Consul.
Not sure how Postgres fits into anything but that’s besides the point.

The terms are a little confusing. Consul’s “datacenter” value is how consul servers know which cluster they are in. So they have to be the same value.

If you’re asking about physical datacenters, then again all the vault clusters needs to be on the same network, gossip traffic is UDP and you don’t want it traversing networks.

If you have the option of AWS or GCP, they have the concept of availability zones, which is a good idea, you can deploy the 5 cluster servers into as many different availability zones that you have access to.

Keep the number of nodes a prime number, 5 is preferred (actually it’s the default minimum but you can get away with 3 if you don’t need good HA).

Hi, can you please elaborate more regarding gossip/UDP? I am exposing these nodes using public IP’s so that services in different networks can use it

Read: Required Ports | Consul by HashiCorp

I’d be very careful about exposing your Consul servers to the internet. There is no reason to have anything but a Loadbalancer on public, the nodes should all be behind that and on the same network.