I now have two k8s clusters, cluster-sh and cluster-sz, and I want to realize that the services of both clusters can be integrated with vault. Here are my steps:
1: The vault service is deployed in the cluster-sz cluster. Then k8s authentication is enabled in the vault service. Configure different paths for cluster-sz and cluster-sh: sh_kubernetes and sz_kubernetes. Each path configures the Kubernetes host, Kubernetes CA Certificate and Token Reviewer JWT attributes of the relevant cluster.
2: Specify when cluster-sz cluster integration vault.hashicorp.com/auth-path: auth/sz_kubernetes
Specify when cluster-sh cluster integration vault.hashicorp.com/auth-path: auth/sh_kubernetes
3: There is no problem with cluster-sz cluster integration, and secret data can be obtained. But the vault-agent-init of cluster-sh has been permission denied.
4：For testing, I also deployed a vault service in cluster-sh, then enabled k8s authentication, and then used this service for integration, there is no problem with cluster-sh cluster integration. But the configuration here is exactly the same as the configuration of sh_kubernetes deployed in the vault service of cluster-sz in the first step.
Question: Different clusters initiate login requests to the corresponding k8s path, which should all pass. I suspect that there is a problem with the three attributes of path configuration k8s, but the operation methods of these two paths are the same.
The error kinda tells me there is a problem with the ServiceAccount on the cluster where Vault server isn’t running (cluster-sh).
If you’re using the Token Reviewer API, you’ll need a ServiceAccount on that (cluster-sh) cluster with the system:auth-delegator ClusterRole bound to it. Besides, you may want to issue a long-lived token for that SA. You’d then need to configure the Vault Kubernetes auth backend with the arguments kubernetes_host, token_reviewer_jwt and kubernetes_ca_cert so that Vault is able to authenticate to the other cluster as that ServiceAccount and make requests to the Token Reviewer API.
Another option would be to configure the Vault JWT auth backend with the Kubernetes OIDC provider instead of the Kubernetes auth backend (which is more suitable for when the server is running on the same Kubernetes cluster).
Yet another option is carry on using the Kubernetes Vault auth method, but to leave token_reviewer_jwt unset and set disable_local_ca_jwt=true … in this mode, Vault uses whatever JWT is presented by the client trying to log in, to invoke the TokenReview API … which means you need to grant every K8s service account the system:auth-delegator role - you’ll need to consider whether you’re happy with that, but it seems pretty safe to me.
Thank you very much, the problem has been solved. The first is that the service account has no binding authority, and the second is that the cluster api server of the cluster-sz cluster has access whitelist control (cluster-sh does not allow access)
