Currently we have 7 Data Centers distributed globally (Los Angeles, Sydney, Tokyo, Amsterdam, London, New York and Frankfurt).
We’re using London for Vault, and it’s a single but very powerful machine.
When we deploy we sometimes need to reload Consul, and when we do this, we get some strange issues with our services. Those that are connected to Sydney, Tokyo and Los Angeles, all pretty far from the ACL DC regularly fail Healthchecks on each Service. One particular service is actually on the same device and is running a TTL healthcheck locally, so really should not be failing.
My question is: Does a greater distance from the ACL DC cause Consul Healthchecks or just Consul in general to run slowly? Should I replicate the ACL DC in a few places and will this fix the issue?