I receive the user requests from our applications users to my ingress via the AWS Load Balancer. However I am having difficulty in sending this request to Consul Mesh (version 0.32.0).
This is my request flow
Browser → AWS LB → nginx-ingress → service → Pod (with connect-inject)
When my service is not under Consul Mesh, I am able to reach the service.
But the moment the deployment for the service is under Consul Mesh (via connect-inject), the service is unreachable (502 error) via the nginx-ingress (though I am still able to access the service through port forwarding to my local system).
If I remove this service from Consul mesh by removing the connect-inject annotations, I am able to reach the service, but then I am not able to reach the other services which are under Consul Mesh.
Service A (Not under Consul Mesh) --> Service B (Under Consul Mesh)
Above sequence is not working either. Though I am able to communicate between services inside the mesh, but not from a Kubernetes service which is outside the mesh.
What is wrong with my configuration?
We’ve just published some high level guidance on this configuration : Ingress Controller Integrations | Consul by HashiCorp
If you’ve already followed this, could you post a little bit more about your configuration such as the annotations you applied to the nginx deployment as well as confirmation that the ingress controller comes online correctly as well?
I followed the documentation you referred to.
I have added the following annotations to the nginx-ingress-controller deployment
After applying this annotations, the nginx-ingress-controller is not coming up. It remains stuck in the init stage. I have seen the documentation talking about this problem, and I think my transparent-proxy-exclude-inbound-ports, and transparent-proxy-exclude-outbound-cidrs are correct too.
On seeing the log of the container consul-connect-inject-init, I see the error
[ERROR] Timed out waiting for service registration: error=“unable to find registered connect-proxy service”
How do I debug as to what is wrong?
I deleted the consul setup and did a fresh setup from beginning. This time I was able to start the ingress controller without any problem. My requests are indeed reaching the intended pods. However path for every request is coming as ‘/’, irrespective of the path in the original request. On enabling the http logging for envoy-sidecar container, I see the path in the envoy sidecar too is ‘/’.
Here are some of the configs I am using
Consul version: 0.32.0
- protocol: TCP
- host: webmagic.dev2.flowmagic.io
- name: REDIS_HOST
- name: WEBMAGIC_UI_HOST
- name: KEYCLOAK_URL
- containerPort: 3000
If the envoy proxy is only getting
/ then I think there’s probably an issue with the Ingress object’s config? Does
nginx.ingress.kubernetes.io/rewrite-target: / work? Looking at Rewrite - NGINX Ingress Controller they are using something a bit different.
Hello @lkysow Thanks for your help. My ingress configuration was indeed a problem. After fixing it, I am able to make it work as expected.