I am trying to configure an Ingress Gateway to use Envoy escape hatch overrides, namely the “envoy_public_listeners_json” flag. In the docs for an Ingress Gateway it says you can use escape hatch overrides, but I am not able to get this to work.
Here is how I am configuring my ingress gateway currently. I am using “consul config write ingress-service.hcl” to write the config file to my consul agent, alongside a service router whose name is “api”. When I try to add an escape hatch to the config file under Proxy.Config I get a “key not found error”.
I’d like to better understand how you’re trying to use the Ingress functionality. Would you mind sharing a bit more detail regarding what you’re trying to accomplish using the escape hatch override?
Hey @blake. I’m working on replacing my API Gateway with an envoy implementation that has dynamic service discovery (thus using Consul). I have a separate offsite external authentication server that will handle authenticating my users.
I am trying to use the Ingress Gateway (IG) as an entrypoint to my service mesh (which holds my numerous registered consul services registered as envoy sidecar proxies --> using consul connect for this). Here is the ideal setup:
Client --> IG Gateway --> [whichever service the client requested].
I was trying to use the envoy.ext_authz filter to handle external authentication but I was not able to add this to the filter chains of the IG Gateway. For this reason I tried switching to this next setup (shown below) where I have a separate envoy front proxy that I configured without consul. All it does is send traffic to an ext auth server for authentication, waits for the HTTP status 200 msg, and then forwards traffic to the IG Gateway.
Client --> Envoy Front Proxy --> IG Gateway --> [whichever services the client requested].
This brings up a security flaw because someone could just directly access the IG Gateway, completely bypassing the front proxy which handles all the authentication. Or, if the user was only authenticated for Service A but then gets to the gateway and calls Service B, they would be able to access it since no authentication is taking place at the Gateway. For this reason I am trying to use the escape hatch to override the listener in the IG Gateway to ONLY take calls from the Envoy Front Proxy. Not sure if this is the best way to go about it.
Thanks for reading through this big wall of texts. Please let me know if I need to explain any parts in more detail. Maybe there is a different kind of setup I could use that would be more ideal for the kind of workflow I am going for?
Has anyone been able to get escape hatch to work with the ingress gateway? Or maybe external authentication working on it (envoy.filters.http.ext_authz) ?
Another request for documentation on how to access the escape hatch for ingress gateways here, please. Current documentation references proxy.config of a service definition, but for ingress gateways (per the tutorial https://learn.hashicorp.com/tutorials/consul/service-mesh-ingress-gateways) we’re using the ingress-gateway “kind” – not a service – so no proxy.config parameter.
Unfortunately ingress gateways currently do not support escape hatch overrides in any capacity. However, I just created a GitHub issue for it so that we can schedule the work: https://github.com/hashicorp/consul/issues/8722
Thank you for providing additional context on your use cases. This will help us when planning a solution.
Without current support for these escape hatch overrides, is there any other way to implement an ext_authz (or a rate limiter, for example) with the current ingress gateway?
@freddygv I’m new to Consul, but is this something that community members such as myself could help to build out to fast-track it?
My use case is that I currently support a hodgepodge of AWS ECS services and apis, some with no security at all, some with an AWS api gateway, as well as a whole slew of Mulesoft APIs, which use the Anypoint api gateway, and on top of that we have a lot of legacy windows services.
I’m evaluating Istio and Consul Connect as possible options to bring all of this together into a coherent, container-based service mesh.
But, it is a bit of a non-starter if I can’t authenticate and rate limit the APIs at the edge using simple header-based api keys (so I don’t need to convince our vendors to rewrite their authentication schemes)
We’re a non-profit, getting hit by cost reductions due to covid eliminating one of our income streams, so buying an off-the-shelf API Gateway like Gloo or Ambassador is not in the cards.
