Exported services across peered cluster are unable to communicate. `destination_peer` parameter in upstream job not accepted

Current Setup

  1. Nomad based consul clusters, peered.
  2. Mesh and mesh-gateways in each of the clusters with ACL enabled and tokens with policies as shown below
service_prefix "mesh-gateway" {  # consul service is registered under the name mesh-gateway.
   policy = "write"
}
service_prefix "" {
   policy = "write"
}
mesh_prefix "" {
   policy = "write"
}
agent_prefix "" {
   policy = "read"
}
node_prefix "" {
   policy = "read"
}
mesh = "write"
peering = "read"
  1. Setup service-intentions, exported-service, proxy-default objects as described here and here.
  2. 2 hello-world service setup as nomad jobs with consul connect along with upstream setup (I think I am facing issue here).
  3. ACL enabled tokens for services to communicate with each other. Attaching policy below
service_prefix "" {
    policy = "read"
}
node_prefix "" {
    policy = "read"
}

Consul info for both Client and Server

agent:
	check_monitors = 0
	check_ttls = 0
	checks = 0
	services = 0
build:
	prerelease = 
	revision = bd257019
	version = 1.14.3
	version_metadata = 
consul:
	acl = enabled
	bootstrap = false
	known_datacenters = 1
	leader = true
	server = true
runtime:
	arch = arm64
	cpu_count = 2
	goroutines = 310
	max_procs = 2
	os = linux
	version = go1.19.4

Operating system and Environment details

OS: Ubuntu ,
Architecture: arm64

Observations:

  1. Mesh gateways are deployed successfully.
  2. Peering via UI is successful. and I can see the exported/imported services in the UI.
  3. Health check apis for exported services pass successfully.
  4. consul catalog services , however doesn’t show the exported services. I am not sure if it should?

I am unable to debug is why the upstream section in my exported services is unable to accept destination_peer parameter. The nomad job simply rejects it while applying. This is how my the sidecar initializes the listener.
[][1][info][upstream] [source/server/lds_api.cc:82] lds: add/update listener demo-app-cluster-2?dc=cluster-2:127.0.0.1:9090

What the connect section looks like:

       "Connect": {
           "Native": false,
           "SidecarService": {
             "Tags": null,
             "Port": "",
             "Proxy": {
               "LocalServiceAddress": "",
               "LocalServicePort": 8081,
               "Upstreams": [
                 {
                   "DestinationName": "demo-app-cluster-2",
                   "DestinationNamespace": "",
                   "DestinationPeer":"cluster-2"
                   "LocalBindPort": 9090,
                   "Datacenter": "cluster-1",
                   "LocalBindAddress": "127.0.0.1",
                   "MeshGateway": {
                     "Mode": "local"
                   }
                 }
               ],
               "Expose": null,
               "Config": null
             },
           },
           "SidecarTask": {...},
           "Gateway": null
         },