Problem: I’m using external vault server which is hosted on seperate k8s cluster. Using pod annotations in my application pod trying to render the secrets. As mentioned in: [Integrate a Kubernetes Cluster with an External Vault | Vault - HashiCorp Learn]
Injector created with external vault
helm install vault hashicorp/vault \
--set "injector.externalVaultAddr=https:myvault.com"
vault secret
VAULT_HELM_SECRET_NAME=$(kubectl get secrets --output=json | jq -r '.items[].metadata | select(.name|startswith("vault-token-")).name')
Enabled vaulth auth
vault auth enable kubernetes
JWT, CA_CRT and k8s Host
TOKEN_REVIEW_JWT=$(kubectl get secret $VAULT_HELM_SECRET_NAME --output='go-template={{ .data.token }}' | base64 --decode)
KUBE_CA_CERT=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --decode)
KUBE_HOST=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.server}')
Kubernetes config with SA, namespace details
vault write auth/kubernetes/config \
token_reviewer_jwt="$TOKEN_REVIEW_JWT" \
kubernetes_host="$KUBE_HOST" \
kubernetes_ca_cert="$KUBE_CA_CERT"
Policy for vault secrets
vault policy write devwebapp - <<EOF
path "secret/data/devwebapp/config" {
capabilities = ["read"]
}
EOF
Role creation for k8s auth
vault write auth/kubernetes/role/devweb-app \
bound_service_account_names=internal-app \
bound_service_account_namespaces=default \
policies=devwebapp \
ttl=24h
When deploy the application using below annotations:
annotations:
vault.hashicorp.com/agent-inject: 'true'
vault.hashicorp.com/role: 'devweb-app'
vault.hashicorp.com/agent-inject-secret-credentials.txt: 'secret/data/devwebapp/config'
But application pod is stuck at Init:0/1 and when i check the vault-agent-init logs:
2021-07-25T06:12:43.359Z [INFO] sink.file: creating file sink
2021-07-25T06:12:43.359Z [INFO] sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
2021-07-25T06:12:43.361Z [INFO] sink.server: starting sink server
2021-07-25T06:12:43.361Z [INFO] template.server: starting template server
2021-07-25T06:12:43.361Z [INFO] auth.handler: starting auth handler
[INFO] (runner) creating new runner (dry: false, once: false)
2021-07-25T06:12:43.361Z [INFO] auth.handler: authenticating
[INFO] (runner) creating watcher
2021-07-25T06:13:43.362Z [ERROR] auth.handler: error authenticating: error="context deadline exceeded" backoff=1s
2021-07-25T06:13:44.363Z [INFO] auth.handler: authenticating
auth.handler: error authenticating: error=“context deadline exceeded” backoff=1s
This is the error it is showing . Any help would be appreciable. spent couple of days on this 
