Terminating Gateways and Azure Redis / Azure Postgresql

Consul
1

I hope someone else has had the same issue, merely for my own sanity sake.

Quick TL/DR

We make use of consul connect sidecars between all our nomad defined services.
Everything works great, except that we need to connect to two external services. Azure-hosted Redis and Postgresql

For this, we defined two terminating gateways, one for each endpoint.

Here is where things go off the rails:

  1. If more than one service needs to talk to Redis, only the service that starts up first can connect via the Terminating Gateway. All other subsequent services fail.
  2. A single service, with a single instance, connects to PostgreSQL via the Terminating gateway. Push a new job definition via Nomad, causing the service to restart, the new version fails to connect. The only way around this is defining the terminating gateway via the consul CLI again
  3. Similar to the above… A single service, with multiple instances, only the first instance can connect.

I tested the above with a self-hosted Redis instance and did not have the issues. At the moment it seems it is only for the instances hosted in Azure.

Any suggestions? Comments? Feedback?

The boring bits:
How I define the external Node:
Definition file, node.json:
{
“Node”: “azure-redis”,
“Address”: “redis-cache.redis.cache.windows.net”,
“Service”: {
“ID”: “azure-redis”,
“Service”: “azure-redis”,
“Port”: 6379
}

curl --request PUT --data @node.json http://127.0.0.1:8500/v1/catalog/register

Defining the service, that references the node:
Definition File, service.json:
Kind = “terminating-gateway”
Name = “redis-termination-gateway”
Services = [
{
Name = “azure-redis”
}
]
consul config write service.json

1 Like
2

@CarelvanHeerden I am also facing a similar issue in connecting to azure paas services like Redis and Postgres. The thing is I also didn’t face any issue when I tried to access the Redis that was deployed as helm chart through the terminating gateway. With azure i am not able to connect itself. I followed the below steps

  1. Registered the external service in consul catalogue
  2. Onboarded that external service into terminating gateway
  3. Added that service as the upstream in the source service
  4. Created the intention between the source and destination service
  5. While accessing getting the below error

Failed to connect to Redis at localhost:6380 with Error: Error: Redis connection to localhost:6380 failed - connect ECONNREFUSED 127.0.0.1:6380