Current Setup
- Nomad based consul clusters, peered.
- Mesh and mesh-gateways in each of the clusters with ACL enabled and tokens with policies as shown below
service_prefix "mesh-gateway" { # consul service is registered under the name mesh-gateway.
policy = "write"
}
service_prefix "" {
policy = "write"
}
mesh_prefix "" {
policy = "write"
}
agent_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
mesh = "write"
peering = "read"
- Setup service-intentions, exported-service, proxy-default objects as described here and here.
- 2 hello-world service setup as nomad jobs with consul connect along with upstream setup (I think I am facing issue here).
- ACL enabled tokens for services to communicate with each other. Attaching policy below
service_prefix "" {
policy = "read"
}
node_prefix "" {
policy = "read"
}
Consul info for both Client and Server
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = bd257019
version = 1.14.3
version_metadata =
consul:
acl = enabled
bootstrap = false
known_datacenters = 1
leader = true
server = true
runtime:
arch = arm64
cpu_count = 2
goroutines = 310
max_procs = 2
os = linux
version = go1.19.4
Operating system and Environment details
OS: Ubuntu ,
Architecture: arm64
Observations:
- Mesh gateways are deployed successfully.
- Peering via UI is successful. and I can see the exported/imported services in the UI.
- Health check apis for exported services pass successfully.
- consul catalog services , however doesn’t show the exported services. I am not sure if it should?
I am unable to debug is why the upstream section in my exported services is unable to accept destination_peer
parameter. The nomad job simply rejects it while applying. This is how my the sidecar initializes the listener.
[][1][info][upstream] [source/server/lds_api.cc:82] lds: add/update listener demo-app-cluster-2?dc=cluster-2:127.0.0.1:9090
What the connect section looks like:
"Connect": {
"Native": false,
"SidecarService": {
"Tags": null,
"Port": "",
"Proxy": {
"LocalServiceAddress": "",
"LocalServicePort": 8081,
"Upstreams": [
{
"DestinationName": "demo-app-cluster-2",
"DestinationNamespace": "",
"DestinationPeer":"cluster-2"
"LocalBindPort": 9090,
"Datacenter": "cluster-1",
"LocalBindAddress": "127.0.0.1",
"MeshGateway": {
"Mode": "local"
}
}
],
"Expose": null,
"Config": null
},
},
"SidecarTask": {...},
"Gateway": null
},