Permission denied when authenticating pod to external vault service running on gke

Vault
1

I am trying to set up a vault on a private cluster and authenticate it from the application cluster.

Vault (Private GKE cluster)

-> vault LoadBalancer 240.130.0.146 10.210.129.104 8200:30357/TCP,8201:32085/TCP (Internal LB)

Application (Private Gke cluster)

I have followed the entire steps mentioned in documentations.

In My vault-inject-injector

K8s authentication

$ export VAULT_SA_NAME=$(kubectl get sa vault-auth -o jsonpath="{.secrets[*]['name']}")
$ export SA_JWT_TOKEN=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data.token}" | base64 --decode; echo)
$ export SA_CA_CRT=$(kubectl get secret $VAULT_SA_NAME -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo) 

# determine Kubernetes master IP address (no https://) via `kubectl cluster-info`
$ export K8S_HOST=<K8S_MASTER_IP>

# set VAULT_TOKEN & VAULT_ADDR before next steps
$ vault auth enable kubernetes
$ vault write auth/kubernetes/config \
        token_reviewer_jwt="$SA_JWT_TOKEN" \
        kubernetes_host="https://$K8S_HOST:443" \
        kubernetes_ca_cert="$SA_CA_CRT"

I am getting following error -

2020-08-26T08:24:59.554Z [ERROR] auth.handler: error authenticating: error="Error making API request.

URL: PUT http://10.210.209.104:8200/v1/auth/kubernetes/login
Code: 403. Errors:

* permission denied" backoff=2.908150444
2020-08-26T08:25:02.463Z [INFO]  auth.handler: authenticating

All setup works fine when I am using single cluster ( vault and application both running on same cluster)