Kubernetes auth fails when istio in the picture

Would appreciate any pointers from anyone using kubernetes auth method with enterprise vault as an external endpoint.

We are testing k8s auth with vault side car on one of our EKS clusters and keep getting permission denied during the login. This only happens if istio injection is enabled on the pod. Without istio injection, the k8s auth method works just fine with vault side car.

Found this old issue thread related to this: vault-k8s and istio service mesh don't work together · Issue #41 · hashicorp/vault-k8s · GitHub

But based on the resolution mentioned in there, we are already using the recommended annotations so vault agent side car comes up before istio-init.

Again, the auth works without istio injection enabled so it looks like something associated with istio in the picture.

vault.hashicorp.com/agent-init-first: "true"
vault.hashicorp.com/agent-inject: "true"

Appreciate any pointers based on your experience with this.

Versions being used:

EKS: 1.29
Vault side car: 1.15.7
Istio: 1.20.3

We figured out the issue. It was related to using IRSA and k8s auth: Injected config tries to use IRSA token instead of the k8s service account token · Issue #544 · hashicorp/vault-k8s · GitHub

The workaround that worked is to use this annotation specified in this patch: Annotation `vault.hashicorp.com/auth-config` values are overridden · Issue #456 · hashicorp/vault-k8s · GitHub

vault.hashicorp.com/auth-config-token-path: /var/run/secrets/kubernetes.io/serviceaccount