Hi, I try to connect my external vault (Hashicorp Vault Platform, starter tier) to kubernetes (EKS) so I could consume data from the external vault in the pods but when I try to start the application with the vault side-car container it stucks in Init:0/1 status.
-------------
APP LOG
-------------
2022-08-25T17:21:16.218Z [ERROR] auth.handler: error authenticating:
error=
| Error making API request.
|
| URL: PUT https://vault-cluster-public-vault-a1879003.49001524.z1.hashicorp.cloud:8200/v1/auth/kubernetes/login
| Code: 403. Errors:
|
| * permission denied
backoff=1m57.65s
2022-08-25T17:23:13.877Z [INFO] auth.handler: authenticating
2022-08-25T17:23:13.952Z [ERROR] auth.handler: error authenticating:
error=
| Error making API request.
|
| URL: PUT https://vault-cluster-public-vault-a1879003.49001524.z1.hashicorp.cloud:8200/v1/auth/kubernetes/login
| Code: 403. Errors:
|
| * permission denied
backoff=3m21.16s
-------------
INJECTOR LOG
-------------
2022-08-25T15:27:28.243Z [INFO] handler.auto-tls: Generated CA
2022-08-25T15:27:28.247Z [INFO] handler: Starting handler..
Listening on ":8080"...
2022-08-25T15:27:28.343Z [INFO] handler.certwatcher: Updated certificate bundle received. Updating certs...
2022-08-25T15:27:28.351Z [INFO] handler.certwatcher: Webhooks changed. Updating certs...
2022-08-25T15:27:28.352Z [INFO] handler.certwatcher: Webhooks changed. Updating certs...
2022-08-25T15:27:28.352Z [INFO] handler.certwatcher: Webhooks changed. Updating certs...
2022-08-25T15:27:28.352Z [INFO] handler.certwatcher: Webhooks changed. Updating certs...
2022-08-25T15:27:28.352Z [INFO] handler.certwatcher: Webhooks changed. Updating certs...
2022-08-25T15:50:33.558Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T16:02:53.020Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T16:05:34.659Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T16:30:29.439Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T17:14:19.475Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T17:18:51.680Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
2022-08-25T17:54:57.203Z [INFO] handler: Request received: Method=POST URL=/mutate?timeout=30s
I always get 403 permission denied despite following this step: Integrate a Kubernetes Cluster with an External Vault | Vault - HashiCorp Learn.
Things I verified:
- JWT that was used to configure vault auth backend is correct
- CA that was used to configure vault auth backend is correct
- The service account assigned to my app that I do a curl from exists
- The service account assigned to my app that I do a curl from is setup correctly in vault auth backend role section with correct namespace
Is there a step I missed?
Screenshot:
