Hi, I have a vault server deployed in a gcp cluster. I enabled and configured kubernetes secret engines with the commands and generate service account token.
❯ vault write <k8s-engine-for-cluster-x>/config \
kubernetes_host=$KUBERNETES_HOST \
kubernetes_ca_cert="$KUBERNETES_CA_CERT" \
service_account_jwt="$SERVICE_ACCOUNT_JWT”\
disable_local_ca_jwt=“true”
❯ vault write <k8s-engine-for-cluster-x>/roles/<role-name> \
allowed_kubernetes_namespaces="kube-system" \
service_account_name=<example-sa> \
token_default_ttl="48h"
❯ vault write <k8s-engine-for-cluster-x>/creds/<role-name> kubernetes_namespace=kube-system
I am able able to successfully generate service account token for about 50min ~ 1hr. After that when i execute the command "❯ vault write /creds/ kubernetes_namespace=kube-system " I got error of unauthorized.
Error writing data to <k8s-engine-for-cluster-x>/creds/<role-name>: Error making API request.
URL: PUT http://<vault-server-ip>:8200/v1/<k8s-engine-for-cluster-x>/creds/<role-name>
Code: 500. Errors:
* 1 error occurred:
* failed to create a service account token for <k8s-engine-for-cluster-x>/creds/<role-name>
But when I reconfigure the vault kubernetes secret engine with the same config data, I was able to generate service account token successfully. And it failed again around 50 mins ~ 1hr like before. I tried to configure kubernetes secret engine for the same cluster-x in another vault server running in another k8s cluster, did not encounter the issue.
I am so confused. Is it sth related to the vault server setup? I’ve read the official doc but had no idea what caused the intermittent failure. Can anyone help? Thanks.