Issue with vault agent injector - pods not getting init container if 1 of 2 injectors restarts

Vault
1

We’ve deployed the vault agent injector into kubernetes and set it to 2 replicas and are using the default value from the helm chart:

  leaderElector:
    enabled: true

Whenever the cluster takes a node offline, where an injector pod lives, it appears that all of our application pods that also restart during this time fail to get an init container injected into the pod, which puts those pods into a CLB situation until they are deleted.

2

Ultimately, the workaround is to set the webhook failurePolicy to Fail from Ignore. This ensures that pods don’t get scheduled until the webhook is successful. And a pod without secrets is generally useless, so this seems like a reasonable workaround.