Consul Inject Sidecar with resources

Consul
1

Hi,

I’m trying to find a way to enabled resources for consul connect Inject sidecars on namespaces that have resource quotas enabled.

I tried using proxyDefaults as below in the helm values but could not get to inject the values into the sidecars. Any help/pointer?

connectInject:
enabled: true
default: false
overrideAuthMethodName: consul-auth
centralConfig:
enabled: true
proxyDefaults: |
{
“resources”:"{ “requests”: { “memory”: “128Mi” }, “limits”: { “memory”: “128Mi” } }"
}

2

Hi, unfortunately this currently isn’t supported. It’s on our backlog.

For now, have you tried setting a Limit Range that will give every container a default limit/request?

1 Like
3

Thank you @lkysow. Yes, Limit Range helps. I can wait for backlog as it would be an ideal solution for my use case.

4

Hello,

I am trying to use Consul Inject functionality, but I have the following issue. The scenario is as follows:

  1. I have a OpenShift namespace/project for which I have LimitRange or/and ResourceQuota enabled. One of them (LimitRange or ResourceQuota) is sufficient to replicate my issue. For now, let’s assume that I only have LimitRange object defined and no ResourceQuota on my OpenShift namespace.
  2. I have a Pod definition with correct Connect Inject annotations, so Connect Inject sidecar containers should be injected into the Pod by MutatingWebhookConfiguration (I use official Consul Helm chart to deploy Consul). The Pod is to be created in the namespace in which LimitRange and ResourceQuota are defined.
  3. When I try to submit the definition of the Pod into OpenShift I am getting the following error:

“Error creating: pods “My_pod_name_here” is forbidden: [minimum memory usage per Pod is 10Mi. No request is specified., maximum memory usage per Pod is 4Gi. No limit is specified., minimum memory usage per Container is 10Mi. No request is specified., maximum memory usage per container is 4Gi. No limit is specified. …”

The part related to Containers is repeated 3 times in the error message - probably due to the fact that Connect Inject injects 3 containers in total including the Init Container.

When I remove the LimitRange (and ResourceQuota if enabled) everything is working properly - i.e. sidecar containers are injected and the Pod is accepted by OpenShift.

I am using LimitRange object with default values for memory request and limit.

I am getting similar exceptions when:

  • only LimitRange is defined (described above)
  • only ResourceQuota is defined (described above)
  • only LimitRange and ResourceQuota is defined

I was reading a lot about sequence/order of admission controllers and I think that a simplified picture of my issue is as follows:

  1. I submit my Pod to OpenShift (with one application/original container only)
  2. LimitRanger admissionController set default values for the application container in my Pod (setting explicit memory request/limit for the application container does not help)
  3. In mutating phase for admission controllers: Connect Inject injects 3 sidecar containers (they have no memory request/limit defined and will not have since LimitRanger has already been executed)
  4. Validation phase for admission controllers: LimitRanger (probably?) checks containers in my Pod to make sure they all have memory requests/limits specified - only one of them has memory request/limit specified. The injected containers have no memory requests/limits specified.
  5. OpenShift (admission controller) denies creating the Pod.

I am using old version of OpenShift 3.11 with Kubernetes 1.11.

Any help on the above would be appreciated.

Thanks in advance for any help.

5

Thanks for the detailed write up @wisniewski-dominik. Looks like we just need to implement this then.

6

Hello,

Please let me know which version of Kubernetes/OpenShift you use. I can not make Connect Inject work when there is a Limit Range or a Resource Quota object defined in a OpenShift namespace.

According to:

Kubernetes version 1.15 solves issues related to invocation order of admission controllers.
Unfortunately, it is not feasible for me to upgrade from OpenShift 3.11/Kubernetes 1.11 at the momemt.

Thanks in advance for your reply.