I am attempting to connect two clusters using service mesh, and appear to be having some issues. It’s not clear to me exactly what the approach is, and where I’m going wrong.
The first cluster works well and as expected. I then attempted to connect a second cluster following the instructions given here:
Again, this worked as expected with the services showing up in the consul UI. The problem was that none of the cross cluster services are able to communicate with each other using the service mesh.
I then started to look at the following for creating a mesh gateway in each of the two clusters: HashiCorp Learn
The configuration is completely different to the first example, to the point where I can’t seem to get anything to now work as expected. In the mesh gateways, there is a new datacentre with a server, in the first example there was only a connection to an external server.
Importantly, I need to implement acls and gossip encryption, which worked fine in the first example, but not at all with mesh gateways.
Does anyone have any guidance?
For reference, here is my current configuration for the second cluster (which doesn’t work):
consul: global: name: consul-test datacenter: dc2 metrics: enabled: true tls: enabled: true caCert: secretName: consul-federation secretKey: caCert caKey: secretName: consul-federation secretKey: caKey federation: enabled: true primaryDatacenter: dc1 ## For production turn on ACLs and gossipEncryption: # acls: # manageSystemACLs: true # bootstrapToken: # secretName: consul-bootstrap-acl-token # secretKey: token # replicationToken: # secretName: consul-acl-replication-acl-token # secretKey: token gossipEncryption: secretName: consul-gossip-encryption-key secretKey: key server: replicas: 1 extraVolumes: - type: secret name: consul-federation items: - key: serverConfigJSON path: config.json load: true exposeGossipAndRPCPorts: true # exposes the server gossip and RPC ports as hostPorts ports: # Configures the server gossip port serflan: # Note that this needs to be different than 8301, to avoid conflicting with the client gossip hostPort port: 9301 securityContext: runAsNonRoot: false runAsUser: 0 client: enabled: true exposeGossipPorts: true # exposes client gossip ports as hostPorts hostNetwork: true dnsPolicy: ClusterFirstWithHostNet join: ["provider=k8s host_network=true kubeconfig=/consul/userconfig/consul-kubeconfig/kubeconfig namespace=\"consul\" label_selector=\"app=consul,component=server\""] extraVolumes: - type: secret name: consul-kubeconfig load: false connectInject: # This method will inject the sidecar container into Pods: enabled: true # But not by default, only do this for Pods that have the explicit annotation: # consul.hashicorp.com/connect-inject: "true" default: false controller: enabled: true meshGateway: enabled: true