Consul Connect, Federation, and TCP/ZeroMQ

Consul
1

Hi, I’m assessing Consul for a service mesh where multiple services are going to be communicating over ZeroMQ.

We plan to have several datacenters (some in k8s, and some on VMs), and the services need to be able to communicate over ZeroMQ sockets (which use TCP for the underlying transport), across datacenters.

I have a few questions about whether this could work with Consul connect:

  1. ZeroMQ doesn’t support TLS natively. Consul Connect appears to send traffic between federated datacenters across mesh gateways, and the services’ traffic is encrypted via mTLS (and maybe uses SNI for routing?). Does TLS need to be supported at the application level (zeromq in this case) for this to work?

  2. Is it possible to configure consul such that services communicating within the same AWS VPC avoid NAT traversal (and therefore don’t incur external data transfer costs). Does this change if there are multiple Consul datacenters within the same VPC? I’m thinking we can have a VM datacenter as well as a k8s datacenter in the same VPC

  3. More generally, I’m wondering how the service discovery works with Connect+K8s. When deploying Consul and consul-registered services with helm charts, I assume each service needs to be connecting to the API of its own datacenter as a consul agent. However, in testing I deployed 2 services with connect on k8s, which registered as services, but not as nodes (and I understand from the docs that the agents are nodes). I was able to get these services communicating via the addresses injected in the environment. Should I be running the consul agent in my application in order to do this ‘properly’?

2

Hi @HartS! :wave:

Yes, SNI is used by mesh gateways to route TCP traffic between services.

You do not need to configure TLS at the application level. TLS encryption/decryption will be handled by the Envoy proxies in the service mesh.

This is the default behavior if the services reside within the same Consul datacenter.

If you’re operating multiple Consul data centers, you can connect them together using mesh gateways. The mesh gateways will need to have an IP address that is routable from the other gateway instances. If you’ve configured your VPC appropriately such that it can route packets between clusters, your traffic won’t need to traverse a NAT gateway.