Error using the vault-agent-init to inject secrets from another namespace

I am trying to use the injector provided by vault in order to load secrets into an image. The helm chart was deployed to the default namespace and when I create a deployment in that namespace, the secrets are successfully loaded. However, it does not work from another namespace which seems very weird to me.

Here is the error I am getting:

==> Vault agent started! Log data will stream in below:

==> Vault agent configuration:

                     Cgo: disabled
               Log Level: info
                 Version: Vault v1.7.0
2021-03-31T15:45:22.940Z [INFO]  sink.file: creating file sink
2021-03-31T15:45:22.940Z [INFO]  sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
2021-03-31T15:45:22.940Z [INFO]  template.server: starting template server
[INFO] (runner) creating new runner (dry: false, once: false)
             Version Sha: 4e222b85c40a810b74400ee3c54449479e32bb9f

[INFO] (runner) creating watcher
2021-03-31T15:45:22.941Z [INFO]  auth.handler: starting auth handler
2021-03-31T15:45:22.941Z [INFO]  auth.handler: authenticating
2021-03-31T15:45:22.941Z [INFO]  sink.server: starting sink server
2021-03-31T15:46:22.941Z [ERROR] auth.handler: error authenticating: error="context deadline exceeded" backoff=1s
2021-03-31T15:46:23.941Z [INFO]  auth.handler: authenticating
2021-03-31T15:47:23.942Z [ERROR] auth.handler: error authenticating: error="context deadline exceeded" backoff=1.93s
2021-03-31T15:47:25.874Z [INFO]  auth.handler: authenticating
2021-03-31T15:48:25.875Z [ERROR] auth.handler: error authenticating: error="context deadline exceeded" backoff=3.2s
2021-03-31T15:48:29.078Z [INFO]  auth.handler: authenticating
2021-03-31T15:49:29.079Z [ERROR] auth.handler: error authenticating: error="context deadline exceeded" backoff=6.19s

On the other side, when I run from the default namespace, I get this instead:

==> Vault agent started! Log data will stream in below:

==> Vault agent configuration:

                     Cgo: disabled
               Log Level: info
                 Version: Vault v1.7.0
             Version Sha: 4e222b85c40a810b74400ee3c54449479e32bb9f

2021-03-31T15:02:02.236Z [INFO]  sink.file: creating file sink
2021-03-31T15:02:02.236Z [INFO]  sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----
2021-03-31T15:02:02.236Z [INFO]  template.server: starting template server
[INFO] (runner) creating new runner (dry: false, once: false)
[INFO] (runner) creating watcher
2021-03-31T15:02:02.237Z [INFO]  auth.handler: starting auth handler
2021-03-31T15:02:02.237Z [INFO]  auth.handler: authenticating
2021-03-31T15:02:02.237Z [INFO]  sink.server: starting sink server
2021-03-31T15:02:04.100Z [INFO]  auth.handler: authentication successful, sending token to sinks

Please help me understand what is going on and how I can go around this. Thank you.

Depending on the auth method you are using, you may be hitting a role that is looking for the namespace you are coming from as part of the claim.

Take a look here:

Cheers,
Grant