Hi @immabird unfortunately ProxyDefaults only apply to specific mesh service proxies (i.e. services that aren’t gateways). There have been some issues created some time ago to allow for adding escape hatch overrides to ingress gateways (API Gateways are custom ingress gateways):
I am trying to configure an Ingress Gateway to use Envoy escape hatch overrides, namely the “envoy_public_listeners_json” flag. In the docs for an Ingress Gateway it says you can use escape hatch overrides , but I am not able to get this to work.
Here is how I am configuring my ingress gateway currently. I am using “consul config write ingress-service.hcl” to write the config file to my consul agent, alongside a service router whose name is “api”. When I try to add an escape hatch to the config …
and the corresponding GitHub issue here:
opened 03:37PM - 22 Sep 20 UTC
type/enhancement
theme/connect
theme/envoy/xds
theme/ingress-gw
Ingress gateways are unable to utilize any of the existing listener escape hatch… overrides.
At the moment, [their upstream config is generated](https://github.com/hashicorp/consul/blob/master/agent/proxycfg/state.go#L1478) from the `GatewayService` type which comes from the ingress gateway config entry. One potential solution is to accept a listener override in the [ingress listener definition](https://www.consul.io/docs/agent/config-entries/ingress-gateway#listeners).
A caveat with using that escape hatch is that we need to document that users need to ensure the [route_config_name is set to the listener port](https://github.com/hashicorp/consul/blob/master/agent/xds/testdata/listeners/ingress-http-multiple-services.envoy-1-12-x.golden#L29), since this is the key we use for RDS routes.
For additional context see this discuss thread: https://discuss.hashicorp.com/t/using-escape-hatch-overrides-with-ingress-gateway/10834
from a corresponding comment in another issue, it looks like we just got this put on an internal roadmap for the core Consul team, but I can’t give you an estimate as to when this would be supported yet:
opened 08:46PM - 06 Jun 22 UTC
closed 03:56PM - 04 Oct 22 UTC
type/question
theme/ingress-gw
Team,
I have a service deployed in a mesh and I am leveraging ingress gateway t… o route traffic from outside to my service.
My service serves HTTP traffic and have long running connections to the service, service can have up to 25k open connections.
I am simulating 20k open connections for performance evaluation and running into this error:
`Sending local reply with details upstream_reset_before_response_started{overflow}`
and receiving 503 response for the request.
Here is a diagram which shows the communication flow:
<img width="970" alt="Screen Shot 2022-06-06 at 3 37 15 PM" src="https://user-images.githubusercontent.com/30634133/172244604-1e9157f0-30a8-4818-a724-712a8141f5a7.png">
As I understand, I would need to update the max connection configuration at two places:
- connect-proxy (running as a sidecar task for Service A)
- Ingress gateway
Here is the configuration that I have tried but no luck with resolving the issue:
- Configuration for connect-proxy:
```
connect {
sidecar_service {
proxy {
config {
local_request_timeout_ms = 0
envoy_local_cluster_json = <<EOF
{
"@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
"name": "local_app",
"type": "STATIC",
"connect_timeout": "5s",
"circuit_breakers": {
"thresholds": [
{
"priority": "DEFAULT",
"max_connections": 10000,
"max_requests": 10000
}
]
},
"load_assignment": {
"cluster_name": "local_app",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 2001
}
}
}
}
]
}
]
}
}
EOF
}
}
}
}
```
I was able to verify the max connection change in config_dump on connect-proxy
- Configuration for ingress-gateway
```
consul config read -kind service-defaults -name ingress
{
"Kind": "service-defaults",
"Name": "ingress",
"TransparentProxy": {},
"MeshGateway": {},
"Expose": {},
"UpstreamConfig": {
"Defaults": {
"Limits": {
"MaxConnections": 10000,
"MaxPendingRequests": 512,
"MaxConcurrentRequests": 10000
},
"MeshGateway": {}
}
}
}
```
config_dump on ingress gateway did not update.
Is this configuration even supported?
If currently not, is there any workaround to first at least test it out?
Appreciate any assistance on this ticket.
Versions:
Consul - 1.10.8
Nomad - 1.2.6
Related ticket:
https://github.com/hashicorp/consul/issues/12373