We have back-end application running in K8s pod providing REST API to other services in K8s. As part of its functionality it needs to create and read KV secrets to/from Vault.
I was wondering what is the best practice to achieve this. After reading the docs I think the vault agent sidecar is a great choice to keep the Vault authentication and Vault token renewal process outside the application concern. However, I have not found the information how the application itself can facilitate the vault agent side car to communicate with vault to create/read secrets.
The docs describe that vault agent container stores the vault token into the file (if sink is configured so), however I have neither found if the application running in other container in the same pod can access this file to use it for Vault API calls nor if this is actually good practice.
Any hints about how to achieve the scenario described above is highly appreciated.
You’re making this more difficult than it needs to be. The sidecar is not only there for auth and renewals it acts as a proxy to the Vault instance. Your application makes the same requests to the proxy and it will either answer (if cached) or get it from Vault with the authentication attached.
Thank you a lot for your answer. I went through the Vault agent caching docs yesterday and some sentences there really indicate what you are saying, however the scenario is not described to more detail. Examples in the docs showing requests from the client code towards the agent (instead of vault directly) would remove any doubts
Let me wrap up with few questions:
Are you saying we do not need to authenticate to Vault from the app if there is a vault agent running (the app will point the vault API requests towards the vault agent)?
Are you saying the authentication to Vault from the application is not needed in this scenario (as the vault agent handles that)?
It is OK to just call the Vault API (vault address pointing to vault agent running) to write/read secrets straight away (Vault token does not need to be in the header X-Vault-Token in the requests towards the vault agent instance)?
Preface … Vault sidecar container, is the vault agent running in a container. I’m just going to talk about the agent running as a process since I’m can’t talk to Kub differences.
Correct, if you leave the agent running exit_after_auth = false then the agent acts as a authorized proxy.
$ vault agent -config ./vault_agent.hcl -log-level=debug -tls-skip-verify
==> Vault agent started! Log data will stream in below:
==> Vault agent configuration:
Api Address 1: http://127.0.0.1:8200
Then I can run, this without authentication to vault locally:
Correct, as long as the agent is still running and can renew it’s auth then your app can talk to the agent and get access to whatever it is allowed per the auth policy.
Vault doesn’t distinguish between application access and “curl” or “vault cli” commands. As long as it’s allowed to accept the request it’ll accept it and act on it. So yes, you can use vault cli or curl to talk to the agent without any other requirements.
I don’t want to make this a huge thing, but this is opening up a hole through which anything can flow. You need to be very secure about how the agent is configured, who has access to it (pod/namespace/host). For example in a VM or container, you don’t want to bind to the public IP, otherwise anyone who has the IP and port can get to your secrets. Caution must be taken, but otherwise it’s a great fast onboarding solution for applications.
