First, what am I trying to achieve?
I want to connect MongoDB instances in Kubernetes over Consul Connect, using the Envoy proxies. On the long run I want to connect them over cluster boundaries in 2 clusters using Consul Federation.
What have I done yet?
I deployed a mongo StatefulSet with a ClusterIP Service per pod and synced them to Consul, so that mongo-0-mongo.service.consul resolves to the same IP as mongo-0.mongo.mongo.svc.cluster.local. Also, the pods each are annotated to inject the Consul sidecar. To complete config, I also added the connect-service-upstreams like mongo-0-mongo:10.
Then I execed into one of the mongo pods and initialized the ReplicaSet.
Where am I failing?
I cannot change the mongo hostname to localhost:10 to connect via Consul service mesh. The error sais there is no node with this address.
To check that my Consul setup is working, I did a test with the server and client pods described here. This works.
I also tested if the mongo-0 pod’s IP can be resolved from inside a simple busybox container that I annotated accordingly: kubectl exec shell -- nslookup mongo-0-mongo.service.consul <consul-dns-ClusterIP>. This works aswell and gives me the pod’s current IP.
What am I doing wrong with the mongo hostname? And: how do the mongo pods communicate in the end? Which port do they use, could this be a firewall problem?
Thank you so much in advance for helping me!