How many times does Vault communicate with an application to inject secrets?

Vault
1

I am currently evaluating whether I should adapt Vault in my cluster. There are a few questions I have in mind right now, it would be great if anyone in the community can help me with them:

  • how many times does Vault communicate with an application/pod to inject secrets? Does that happen only when the pod is created, or when the secret is changed?

  • do I need to rollout restart the pod manually when the secret has changed?

2

Vault itself never initiates communication with your applications/pods.

Any such injection is a result of one or more separate add-ons that bridge between Vault and the Kubernetes ecosystem. You’d have to make it clearer about which one you’re looking at, for people to answer questions about its behaviour.

3

Sorry for making an unclear question. I am referring to the guide, which inject secrets into a Sidecar container. In that case, would changing secrets make deployment restart? And is the injection only made when the pod/deployment is initialized?

4

I’m not particularly familiar with this approach, never having directly used it myself.

But, I do know enough to point out that the secrets themselves are not being injected.

What is happening, is a sidecar container running vault agent ... is being injected, and it is the process in that sidecar which is reaching out to the Vault server according to its own configuration.

From this, you can infer that:

  • Changing secrets is never going to make the pod restart, unless it is deliberately built to detect the change itself and exit.

  • Injection of the vault agent process only occurs when the deployment is being created. But what the agent process does afterwards will depend on how it is configured.

5

In addition to what @maxb said, you can configure how often the agent running as a sidecar polls Vault for updated KV secrets (for leased secrets, it will depend on the lease’s TTL).

The interval can be specified with the Pod annotation vault.hashicorp.com/template-static-secret-render-interval

Your application needs to detect the change in the file where Vault agent saves the secrets and deal with it accordingly.

6

On the topic of Rollout Restarts, you have basically two options when using the Agent Injector to inject vault secrets into your pods:

  • Instrument your application to notice when the secret has changed via some file-watcher and have it trigger whatever internal process you need to manage a secret change.
  • You can use this annotation with your application to do something, either hit an endpoint in your application or cause it to restart in a number of ways (kill the container, for example). Agent Sidecar Injector Annotations | Vault | HashiCorp Developer

One final alternative is the new Vault Secrets Operator Vault Secrets Operator | Vault | HashiCorp Developer which went GA today. While it is a different approach to “injecting” secrets it is a substantially lighter weight tool that has built-in graceful secret lifecycle management via automating rollout restarts on secret changes or expiry of certs/leases.

1 Like