Plugin CNI : Privilege escalation & not read-only root filesystem error on connect-inject-init and dataplane sidecar

Consul
,
1

hello,

I am deploying consul on kubernetes, gatekeeper is installed with policies which catch Privilege escalation and access to root file system with root user and block injection.
I am not administrator of kubernetes and I can’t manage policies.

But in order to test pluging CNI, I got a namespace exampted of gatekeeper’s control policies.

So I try to deploy with CNI plugin to avoid usage of privilege escalation and access to root file system with root user by consul-connect-inject-init.

# values.yaml
----
connectInject:
  enabled: true
  transparentProxy:
    defaultEnabled: true
  cni:
    enabled:  true
    logLevel: debug
    namespace: reserved-consul-cni
  k8sAllowNamespaces:
  \- consul-dev
controller:
  enabled: true
global:
  acls:
    createReplicationToken: true
    default_policy: deny
    down_policy: extend-cache
    manageSystemACLs: true
  datacenter: kube-dev-dc1
  federation:
    createFederationSecret: false
    enabled: false
  image: hashicorp/consul:1.14.0
  imageConsulDataplane: hashicorp/consul-dataplane:1.0.6
  imageK8S: hashicorp/consul-k8s-control-plane:1.0.10
  imagePullSecrets:
  \- name: artifactory-cred
  logJSON: true
  logLevel: debug
  name: consul
  serverAdditionalDNSSANs:
 \- consul-server.consul.svc.cluster.local
  tls:
    enableAutoEncrypt: true
    enabled: true
    verify: true

server:
  replicas: 3

ui:
  enabled: true
  service:
    port:
      http: 80
      https: 443
    type: ClusterIP

I deploy consul on consul-dev namespace were is applied gatekeeper policies
and I deploy consul CNI plugin on reserved-consul-cni exampted to follow gatekeeper policies.

Deploment is ok.

cluster versions:

kubernetes v 1.22.9
helm chart 1.0.10
image: hashicorp/consul:1.14.0
imageConsulDataplane: hashicorp/consul-dataplane:1.0.6
imageK8S: hashicorp/consul-k8s-control-plane:1.0.10

Unless I’m mistaken, following documentation, if I use CNI plugin, consul-connect-inject-init will not need to use root user and read/write root filesystem, ridht?

from documentation, PR and docs :

In some place I read :

By default, Consul generates a connect-inject init container as part of the Kubernetes Pod startup process. The container configures traffic redirection in the service mesh through the sidecar proxy. To configure redirection, the container requires elevated CAP_NET_ADMIN privileges, which may not be compatible with security policies in your organization.

so Privilege escalation and read-write root filesystem shouldn’t be needed?

When I try to deploy static-client and server as described here in tutorial I get that (like I didn’t deployed consul CNI pluging) :

admission webhook "validation.gatekeeper.sh" denied the request:
[psp-allow-privilege-escalation-container] Privilege escalation
container is not allowed: consul-connect-inject-init

[psp-allow-privilege-escalation-container] Privilege escalation
container is not allowed: consul-dataplane

[psp-readonlyrootfilesystem] only read-only root filesystem container is
allowed: consul-connect-inject-init

for dataplane I note that, I must to deploy consul 1.2.2 because in this release “control-plane: When using transparent proxy or CNI, reduced required permissions by setting privileged to false.”(Release v1.2.2 · hashicorp/consul-k8s · GitHub)

But for consul-connect-inject-init, what is wrong ?

Thank you in advance for your help.

Best regards.

2

Hi @MiladiHedi,

The reason why you are having this issue is due to the containers not having the securityContext.ReadOnlyRootFilesystem and securityContext.AllowPrivilegeEscalation assigned explicitly by the mutating webhook controller.

This should be a feature request for Consul-K8S.

3

Hi @MiladiHedi, this was fixed in consul-k8s by PR Set ReadOnlyRootFilesystem and AllowPrivilegeEscalation to false by lkysow · Pull Request #3498 · hashicorp/consul-k8s · GitHub and will be available in the next version which should be released tomorrow.

4

Hello @blake ,

Thank you, I checked this tuesday on PR list to check if there was a PR about this subject,
I was surprised to see that a PR was created to fix my issue this monday.
@Ranjandas and @lkysow, thank you for answere and PR.

Can I know, do you will release it in other versions? like 1.16.5 ( now our more recent k8s cluster is on 1.24.11 version and we use 1.16.4, with helm 1.2.4)

Thank you for your help.

Regards,
Hedi.

5

Hi @MiladiHedi,

This fix is now available in consul-k8s 1.1.9, 1.2.5, and 1.3.9.

You should be able to upgrade to version 1.2.5 of the Helm chart to obtain this change.

6

hello,

I confirm it works.

Thank you