Vault: login unauthorized due to: Post apis/authentication.k8s.io/v1/tokenreviews

Vault
1

Hi all,
I am trying to have a pod authenticate to Vault using Kubernetes auth. I was able to successfully inject my secrets into the pod in the test environment before, but now I am getting an authentication error while doing this in the production environment.
When I try and authenticate, I get the following error:

Vault Logs:
[ERROR] auth.kubernetes.auth_kubernetes_c5b8ece7: login unauthorized due to: Post “https://K8S:6443/apis/authentication.k8s.io/v1/tokenreviews”: dial tcp 10.x.x.x:6443: i/o timeout\n"

Kubernetes Logs:
Error making API request.
URL: PUT https://vault_addr:8200/v1/auth/kubernetes/login
Code: 403. Errors:

  • permission denied

Can you help with this or suggest things I should check?

2

Your Vault appears to be unable to connect to your Kubernetes API server.

This is the first thing you must investigate and fix, to proceed.

3

@maxb thanks for your reply. In addition, when I run vault in debug mode, the log below appears. Do you have another comment on this?

[DEBUG] auth.kubernetes.auth_kubernetes_e0453d01: failed to read local service account token, will use client token: error=“open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory”

4

The significance of this message is that Vault is saying:

  • My administrator has not configured me with a token_reviewer_jwt (see API docs for auth/kubernetes/config)
  • Nor is there a local token present where there usually would be if I was running inside Kubernetes
  • My last resort is to re-use the Kubernetes JWT that a Vault client presents when trying to log in, and hope that has permissions to the tokenreviews API
1 Like