Vault-secrets-operator permission denied

I was trying to setup vault-secrets-operator with kubernetes authentication by deploying it via Helm chart (with default values) and following the configuration in the demo, but I get the following permission denied error:

* permission denied     {"type": "Warning", "object": {"kind":"VaultStaticSecret","namespace":"default","name":"test-secret","uid":"575ff59d-a02f-4751-887b-af8817005da2","apiVersion":"secrets.hashicorp.c
om/v1alpha1","resourceVersion":"13045869"}, "reason": "VaultClientConfigError"}                                                                                                                            
2023-05-17T18:03:16Z    ERROR   Reconciler error        {"controller": "vaultstaticsecret", "controllerGroup": "secrets.hashicorp.com", "controllerKind": "VaultStaticSecret", "VaultStaticSecret": {"name":"test-secret","namespace":"default"}, "namespace": "default", "name": "test-secret", "reconcileID": "8b787497-6dd6-4f0c-8c8e-ef1912906232", "error": "Error making API request.\n\nURL: PUT https://XXX/v1/auth/kubernetes/login\nCode: 403. Errors:\n\n* permission denied"}  

My Vault configuration looks like this:

resource "vault_auth_backend" "kubernetes" {
  type = "kubernetes"
}

resource "vault_kubernetes_auth_backend_config" "kubernetes" {
  backend                = vault_auth_backend.kubernetes.path
  kubernetes_host        = var.eks_api_server
  disable_iss_validation = true
}

resource "vault_kubernetes_auth_backend_role" "test" {
  backend                          = vault_kubernetes_auth_backend_config.kubernetes.backend
  role_name                        = "test-role"
  bound_service_account_names      = ["default"]
  bound_service_account_namespaces = ["default"]
  token_period                     = 120
  token_policies = [
    vault_policy.test.name,
  ]
  audience = "vault"
}

resource "vault_policy" "test" {
  name      = "vso-test"
  policy    = <<EOT
path "secret/data/kubernetes/test" {
  capabilities = ["read"]
}
EOT
}

These are my VaultConnection, VaultAuth & StaticSecret configurations:

---
apiVersion: secrets.hashicorp.com/v1alpha1
kind: VaultConnection
metadata:
  namespace: default
  name: default
spec:
  address: https://XXX
---
apiVersion: secrets.hashicorp.com/v1alpha1
kind: VaultAuth
metadata:
  namespace: default
  name: default
spec:
  vaultConnectionRef: default
  method: kubernetes
  mount: kubernetes
  kubernetes:
    role: test-role
    serviceAccount: default
    audiences: ["vault"]
---
apiVersion: secrets.hashicorp.com/v1alpha1
kind: VaultStaticSecret
metadata:
  namespace: default
  name: test-secret
spec:
  vaultAuthRef: default
  mount: secret
  type: kv-v2
  name: kubernetes/test
  refreshAfter: 60s
  destination:
    create: true
    name: test-secret

The namespace, connection reference, service account, role & audience are all correct, because if I change them to an incorrect value, I get a more specific error. I am not sure what the missing piece is.

This could mean various things. You should ensure the Vault server’s log level is set to DEBUG or TRACE, and check the logs for more clarity on why permission is denied.

Possible issues include the service account JWT being malformed, or more likely, it being rejected by the Kubernetes token review API, or problems calling that API.