Consul k8s go template in proxy defaults

Consul
,
1

Hi,

We are trying to use the envoy_local_cluster_json escape hatch in consul proxy defaults. Consul is not setting circuit breaker on the local_app cluster, so we need to override the envoy_local_cluster_json.

We are using the proxy defaults CRD with the following manifest:

apiVersion: consul.hashicorp.com/v1alpha1
kind: ProxyDefaults
metadata:
  name: global
  namespace: consul
spec:
  config:
    envoy_local_cluster_json: "{\"@type\":\"type.googleapis.com/envoy.config.cluster.v3.Cluster\",\"name\":\"local_app\",\"type\":\"STATIC\",\"connect_timeout\":\"5s\",\"circuit_breakers\":{\"thresholds\":[{\"max_connections\":10000000,\"max_pending_requests\":10000000,\"max_requests\":10000000}]},\"load_assignment\":{\"cluster_name\":\"local_app\",\"endpoints\":[{\"lb_endpoints\":[{\"endpoint\":{\"address\":{\"socket_address\":{\"address\":\"127.0.0.1\",\"port_value\":{{ .ServicePort }}}}}}]}]}}\n"
  transparentProxy:
    dialedDirectly: false

Note the {{ .ServicePort }}, we are trying to use go templates to have a dynamic service port as it was done in this issue: Consul connect configure envoy circuit breakers · Issue #7211 · hashicorp/consul · GitHub

However, the above is not working and envoy does not generate a local_app cluster. Is this only possible with HCL or am I doing something wrong?

2

Here is the beautified json for an easier read :slight_smile:

{
  "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
  "name": "local_app",
  "type": "STATIC",
  "connect_timeout": "5s",
  "circuit_breakers": {
    "thresholds": [
      {
        "max_connections": 10000000,
        "max_pending_requests": 10000000,
        "max_requests": 10000000
      }
    ]
  },
  "load_assignment": {
    "cluster_name": "local_app",
    "endpoints": [
      {
        "lb_endpoints": [
          {
            "endpoint": {
              "address": {
                "socket_address": {
                  "address": "127.0.0.1",
                  "port_value": {{ .ServicePort }}
                }
              }
            }
          }
        ]
      }
    ]
  }
}
3

Hi @FelipeEmerim,

After looking at the issue comment you linked, it sounds like that individual is using a custom injector that likely interpolates the Go template prior to providing that configuration to the proxy. I am not aware of any native capabilities in Consul to support this type of templating when using the escape hatch.

4

We found that out by looking at the source code earlier today, but it is good to have a confirmation because we could be missing something.

We opened an issue on consul-k8s since it does not seem like we can fix this problem with escape hatches and the behavior of circuit breakers in the local cluster is affecting many of our services.

I believe we can close this topic, as the question was answered.

Thank you for the support!