I'm getting 403 permission denied when trying to log in to a external Vault with a prokected token from k8s

Vault
1

I’m getting 403 permission denied when trying to log in to a external Vault with a prokected token from k8s.

  • I have an external Vault

  • I’ve setup kubernetes auth according to the official doc as follows:

    vault write auth/kubernetes/config \
    kubernetes_host=“https://kubernetes.default.svc.cluster.local:6443” \
    kubernetes_ca_cert=@ca.crt

  • The https://kubernetes.default.svc.cluster.local URL is reachable despite the questionable name (I’ve set up the resolver/nameserver accordingly) and I see the actual TCP exchange in in tcpdump via tcp/6443 port

  • The application pod is running under the SA default, this account has the clusterrole system:auth-delegator as the doc says.

  • I’ve set up the role as

vault write auth/kubernetes/role/prod
bound_service_account_names=default
bound_service_account_namespaces=prod
policies=prod
ttl=20m

  • I can log in to vault from vault cli using a legacy token bound to a default SA in NS prod.

  • I’ve extracted the projected token from application pod, but when trying to

vault write auth/kubernetes/login 
role=prod 
jwt="<extracted token>"

I got

Error writing data to auth/kubernetes/login: Error making API request.

URL: PUT https://127.0.0.1:8200/v1/auth/kubernetes/login
Code: 403. Errors:

permission denied

and I really cannot understand why or how to diagnose the reason.

I’m seeing similar threads here but without solution, so could please anyone point me to the right direction ?

  • I’ve also tried thhe configuration with dedicted reviewer SA - usin’t it’s token in the knob
vault write auth/kubernetes/config 
kubernetes_host=“https://kubernetes.default.svc.cluster.local:6443” 
kubernetes_ca_cert=@ca.crt 
token_reviewer_jwt="…”

but this gives me the same “403 permission denied errors” when trying to log in either from pod or from vault cli with a projected token.

2

Hi ,

Follow this discussion and steps mentioned in this discussion

HashiCorp Discuss:

Thanks,

Chirag Jain