Sidecar injected pod takes 90+ seconds to become ready

Consul
1

Hi,

We have an issue in our Kubernetes cluster where pods that have the sidecar injected cannot make connections before waiting at least 90 seconds for envoy to become ready. We would like to know if this is expected, and if there is something we can do to reduce the rollout time.

We are using these resources in the sidecar container:

resources:
    limits:
      cpu: 125m
      memory: 128Mi
    requests:
      cpu: 125m
      memory: 128Mi

Do they have to be increased as our cluster gets bigger? Now we have about 200 pods and 11 nodes.

Clarification: When I say that envoy is not ready, I mean that every connection attempt to other resources fail with connection refused. The container status in k8s is green.

2

That’s not expected. Can you run Envoy at debug level and get us the logs on startup?

annotations:
  consul.hashicorp.com/envoy-extra-args: '--log-level debug'
3

Sorry for the delay, I didn’t get a notification when you replied. I’ve done what you mentioned, here are the logs. I changed some names to avoid exposing our services info. I stopped collecting logs as soon as our connections started to succeed.

sidecar.txt (662.9 KB)