East-west traffic with Connect & Docker

Hello !
I’m giving a try to Consul Connect and it works great. The thing I’m currently not able to figure out is how to avoid “east-west” traffic on the specific host, especially in the case of Docker containers.

Is there some documentation about best practices ?

here’s a few things I tried :

  • use host network for the container and run its proxy from the host
    – result : every container seems to be able to use the proxy of one another by sending requests on the right port

  • use separate bridge networks and bind each proxy (still on host) on the specific interface
    – result : if a container is able to figure out the IP of the other network bridge, the traffic is routed to host, then following the host’s routing table, ends on the other interface and traffic is forwarded to the proxy

  • use the default bridge network for containers then run each proxy (as a container this time) in the container’s network (–network=container:my_container_name) then bind the client to both the host’s lo and to the docker bridge IP
    – result : it kind of work to protect outgoing traffic but if I do the same thing for the destination service, I’m unable to reach it…

Hey @err0r500 I am having the same issue that you are experimenting, this is my repo if you take a look https://github.com/Crizstian/cinema-microservice-in-GO/tree/step6-consul-connect/deploy/docker-compose/consul-connect-example

and I am not able to connect to destination service, I am still figuring out what is missing

also you can take a look at Nick Johnson repo, which works without issue:

Hope it is helpful those repos

Hi @err0r500

I just recently went through the same process and ultimately ended up with a docker-compose configuration very much like what @Crizstian has shared. Using the network_mode: service:<app> in the envoy sidecar container definition so envoy and the app share the same network namespace and can reach each other on 127.0.0.1.
We are also currently deploying a consul instance (containerized) per service, but am still trying to figure out if that’s the best approach.

Thanks a lot to both of you !

My current problem is for traffic between 2 hosts : I can see a request leaving the source host attempting to reach the destination using the dest VM IP at the proxy port, so it fails : since the proxy is bound as a side car to the docker service, it’s only reachable on the “docker” IP of the service and can’t be reached “as is” from the outside.

I’ll try to do the opposite, it might work : mount the service as a sidecar of the proxy and expose the proxy port (I’ll keep the rest intact)…

Otherwise I think the solution may come from a network overlay like Flannel or Callico like on k8s but it means assigning CIDR ranges per docker bridge on each host and some other stuff that would defeat the simplicity of the solution I’m looking for…

I’ll keep you updated.

It seems weird, but you need to expose/publish the proxy port on the service container. (Docker won’t let you publish ports on a container using network_mode: service:foo anyway.)
That way traffic from the rest of the network will be able to reach the envoy proxy public listener.

The “opposite” I was talking about in my previous comment actually worked. I wrote a little thing about it : https://matthieujacquot.com/post/zero-trust-network-with-consul-connect-and-docker/

@sl1pm4t thanks a lot, yes since they share the interfaces it will work too.
Your suggestion is when using docker-compose, right ?