Envoy used by service mesh not able to send metrics to sink on the host machine

Background:
We have telegraf running on the host machine as the sink for metrics for all the services (nomad, consul, vault as well as containers ran by nomad).

The problem is that when we setup the service mesh, we want to have enovy proxy also emit its metric to the telegraf agent on the host network.

What we are able to get it working is to hardcode the host ip address in the envoy_dogstatsd_url:

connect {
  sidecar_service {
    proxy {
      config {
        envoy_dogstatsd_url = "udp://172.16.71.143:8125"
      }
}

But what we really want to have is a dynamic ip in there, like:

connect {
  sidecar_service {
    proxy {
      config {
        envoy_dogstatsd_url = "udp://${attr.unique.network.ip-address}:8125"
      }
}

to be able to render the host ip into the job file dynamically. But this is not something we can do today.

Would this be a valid request to add variable templating into that field?

1 Like

Adding a little more here. We wanted to actually run telemetry as a connect enabled service but the problem is that we use UDP for Statsd - which is currently not supported in the Consul Connect system (it is in envoy) - so if we enable that then, while envoy side car can technically talk to the telegraf system, it can’t actually do anything with the traffic.

That leaves our only option to monitor envoy as @fredwangwang has mentioned above. We need the connect sidecar proxy to send its metrics to the vm so the telegraf system can process it.