I am attempting to connect two clusters using service mesh, and appear to be having some issues. It’s not clear to me exactly what the approach is, and where I’m going wrong.
The first cluster works well and as expected. I then attempted to connect a second cluster following the instructions given here:
Again, this worked as expected with the services showing up in the consul UI. The problem was that none of the cross cluster services are able to communicate with each other using the service mesh.
I then started to look at the following for creating a mesh gateway in each of the two clusters: HashiCorp Learn
The configuration is completely different to the first example, to the point where I can’t seem to get anything to now work as expected. In the mesh gateways, there is a new datacentre with a server, in the first example there was only a connection to an external server.
Importantly, I need to implement acls and gossip encryption, which worked fine in the first example, but not at all with mesh gateways.
Does anyone have any guidance?
For reference, here is my current configuration for the second cluster (which doesn’t work):
consul:
global:
name: consul-test
datacenter: dc2
metrics:
enabled: true
tls:
enabled: true
caCert:
secretName: consul-federation
secretKey: caCert
caKey:
secretName: consul-federation
secretKey: caKey
federation:
enabled: true
primaryDatacenter: dc1
## For production turn on ACLs and gossipEncryption:
# acls:
# manageSystemACLs: true
# bootstrapToken:
# secretName: consul-bootstrap-acl-token
# secretKey: token
# replicationToken:
# secretName: consul-acl-replication-acl-token
# secretKey: token
gossipEncryption:
secretName: consul-gossip-encryption-key
secretKey: key
server:
replicas: 1
extraVolumes:
- type: secret
name: consul-federation
items:
- key: serverConfigJSON
path: config.json
load: true
exposeGossipAndRPCPorts: true # exposes the server gossip and RPC ports as hostPorts
ports:
# Configures the server gossip port
serflan:
# Note that this needs to be different than 8301, to avoid conflicting with the client gossip hostPort
port: 9301
securityContext:
runAsNonRoot: false
runAsUser: 0
client:
enabled: true
exposeGossipPorts: true # exposes client gossip ports as hostPorts
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
join: ["provider=k8s host_network=true kubeconfig=/consul/userconfig/consul-kubeconfig/kubeconfig namespace=\"consul\" label_selector=\"app=consul,component=server\""]
extraVolumes:
- type: secret
name: consul-kubeconfig
load: false
connectInject:
# This method will inject the sidecar container into Pods:
enabled: true
# But not by default, only do this for Pods that have the explicit annotation:
# consul.hashicorp.com/connect-inject: "true"
default: false
controller:
enabled: true
meshGateway:
enabled: true