Vault and kubernetes 1.21


We upgraded kubernetes nodes to 1.21 and with that our Vault secret promotion stopped working. We were running somewhat outdated 1.6.0 version which I upgraded to 1.9.4 reading docs.

Now that “iss” error is gone I’ve run into more problems.

What is the suggested way to make Vault work with k8s 1.21?

I get errors like:

403 - permission denied
500 - service account name not authorized

Most pods get this error now:

Error writing data to auth/kubernetes/login: Error making API request.
URL: PUT http://vault.system:8400/v1/auth/kubernetes/login

Code: 403. Errors:
* permission denied

I’ve tried attaching new service accounts and secrets but whatever I do, I can’t make vault work. Actually vault as it self works fine. I can authenticate to it, open Web UI, manage secrets and such but vault-deployer and vault-controller have problems logging in. Also another deployment is not running since it can’t auth to vault.

I would appreciate any help on how to fix current setup.

Kubernetes has changed the possibilities on how service account tokens can be obtained since 1.21. I would like to refer you to the documentation to find out about these changes and how to make your cluster compatible!
At least for now it seems to me like your issue can be fixed by setting disable_local_ca_jwt to true. But please verify with the documentation if this is correct.