We recently added Vault Agent Injector in our K8s(OKE) cluster and started injecting secrets successfully. We have found that metrics server in the cluster is not able to scrape the pods with the vault injector annotations which blocks us in horizontal pod autoscaling. HPA is dependant on metrics from metrics server for autoscale.
metrics server version: 0.6.1
OKE K8s version: 1.22
*Vault versions tried : *
- App Versio: 1.11.2, 1.11.3*
- Chart version: vault-0.21.0, vault-0.22.0*
I believe it is working on k8s 1.23 but metrics server has some other issues with 1.23.
Steps to reproduce:
On 1.22 cluster
- Deploy vault agent injector
- Enable and configure kubernetes authentication in vault
- Deploy pods/deployment with vault agent injector annotations
vault.hashicorp.com/agent-inject: 'true' vault.hashicorp.com/role: 'oke-app-role' vault.hashicorp.com/agent-inject-secret-app.env: 'secrets/app_env'