Unable to get terminating gateway with tproxy working on mesh running on VM

Consul
1

I have a working setup of consul service mesh using tproxy on virtual machines. I am able to reach to service running a VM using service.virtual.consul address from another VM that is in the mesh.

I want to now connect to the external service and make it accessible inside the mesh as external.virtual.consul.

This is what I have done so far,

  1. Added a config entry for terminating gateway as
Kind = "terminating-gateway"
Name = "managed-service-gateway"

Services = [
  {
    Name = "external-service"
  }
]
  1. Register the external service as
{
  "Node": "external_service_node",
  "Address": "external-service.com",
  "NodeMeta": {
    "external-node": "true",
    "external-probe": "true"
  },
  "Service": {
    "ID": "external_service",
    "Service": "external-service",
    "Port": 3306
  }
}
  1. On a new virtual machine, I have installed consul agent, have registered it and on that machine envoy proxy is running as
consul connect envoy -gateway=terminating -register -service managed-service-gateway -- -l debug --log-path /var/log/consul/envoy_logs.txt

After all of this, I could see on the consul dashboard that the external service is now linked under the terminating gateway and in service mesh with the gateway.

When I do run dig external-service.virtual.consul, I could see a virtual IP assigned however, when I try to connect to the external service using mysql -ufoo -h external-service.virtual.consul -p, connection is hanging up with an error “Lost connection to MySQL server at ‘reading initial communication packet’, system error: 0”. However I can connect to the underlying external service without any problems outside of the mesh.

I’ve tried adding with and without redirect-traffic on terminating gatway’s VM but no help. What am I missing?

2

Where are you doing mysql connection test from? If you tried it directly from the terminating gateway VM, it won’t work.

You would need a downstream application that connects to external-service, via the terminating gateway.

3

I am trying to connect it from inside the VM of another service. This VM has a consul agent and envoy proxy running along with firewall redirection rules applied.

My overall setup

  1. There are 3 services registered - 2 normal services lets call it foo and bar, 1 external service external-service which is MySQL.
  2. foo and bar are running on 2 independent VMs. These VMs have consul agent and envoy proxy running and have the firewall redirection rules applied using redirect-traffic.
  3. From foo’s VM, I can make API call to bar using curl bar.virtual.consul, however, from the same VM I am unable to connect to MySQL
4

Update:

Along with MySql, I also registered another external service that works on http. Lets call it baz. I followed the same steps as registering it via v1/catalog/register api and then updated the terminating gateway config entry to include baz service.

This time, when I hit curl baz.virtual.consul from foo’s VM, it worked! That implies for some reason, http based traffic is working fine.

Probably it has to do with proxy defaults? Here is my proxy default config

Kind      = "proxy-defaults"
Name      = "global"
MutualTLSMode = "permissive"
Mode = "transparent"

TransparentProxy {
  DialedDirectly = false
}

Config {
  protocol = "http"
}

Expose {
  Checks = true
}
AccessLogs {
  Enabled = true
  DisableListenerLogs = false
  Type = "file"
  Path = "/var/log/consul/envoy-access-logs.txt"
}

Is the problem this line? - protocol = "http"

5

Found it. Yes, it was happening due to protocol = "http". Changing it to protocol = "tcp" worked.