I have Node JS app inside pods, which need to read vault secrets. So far I found 2 methods for doing that
Using init container to mount secrets as .txt files and read/parse them in my app
Using node-vault connect to vault server directly and read secrets, which requires initial token
For (1) I found this article, where the author is considering it as not secure and complex.
Secrets mounted as volumes are unwieldy—secrets can be stored as environment variables or mounted as a volume. The former technique is widely agreed to be less secure. If you opt for volumes, things quickly get complex when you have a large number of keys. Kubernetes creates one file per key, and you need to read all these files from within the application. There are workarounds, but they can be equally complex.
For the (2) I am not sure how to automatically unseal and obtain initial tokens without using Amazon secrets or other services. Of course I can’t hard code initial token or roll_id/secret_id . What is the typical way of obtaining and passing initial token to the pods?
There will be an update to the CSI driver soon that will go more towards “beta” than “experimental” - but I think its due to the way the CSI secret store is advertised by Microsoft vs the actual functionality.
I would use it in production, and many Vault users are. But thats just my opinion.
Yes I did and check that article and in my original post (1) is exactly referring to that way. However, I would also like to know about option (2), using token. How to obtain, pass and store safely that token inside kubernetes.
You can use the Agent injector to present a token that the application can read (seen as a file to the application). Alternatively you could pass in AppRole details during deployment (again via Kubernetes secrets) which the application can then read and use to authenticate to Vault.
@stuart-c do you mean to map root token located at auth/token path? If so, then it’s agent-inject-token representing root token and it’s not secure to use it as far as I understand.
As for AppRole, then I would have to keep role_id/secret_id pair as Kubernetes secret, which is also not secure, since if Kubernetes secret get compromised, then role_id/secret_id can be used to read secrets. Therefore I am trying to get away from Kubernetes secrets.
You shouldn’t use a Vault root token - ideally all root tokens would be revoked and only created during short periods they are needed for configuration.
What situation are you trying to guard against? Kubernetes secrets (or service accounts if using the Kubernetes auth method) would be the standard way to handle secrets within Kubernetes. If Kubernetes itself it compromised then you should assume all hosted applications are similarly exposed.
@stuart-c correct, but then which token to use for passing to pod through mapped file, as you mentioned before? how to obtain that token, what is the name of that token? could you bring an example of annotations I have to write in order to pass that token?
You would use a token with permissions specific to the application. You could use the Vault injector to create the token or you could use your deployment system (e.g. Terraform, Ansible, Helm, etc.). Althernatively using AppRole or Kubernetes authentication might be easier.
@stuart-c so I am running into a similar dilemma. I am running vault inside a k8s cluster, and I’d like to provision my application with only the role_id for the AppRole I created.
I’ve been trying to get the init container to populate a response wrapped secret_id into my container, so that my application can then login using AppRole, and fetch whatever secrets it needs.
But I’ve looked through the API for agent-inject/agent/annotations.go under your vault-k8s repo and cannot seem to find an annotation dedicated to fetching a secret_id.
It seems as if the inject init or sidecar container is designed to inject secrets directly into the application by sharing the /vault/secrets volume, with an option to authenticate using AppRole and a k8s service account.
But I want my application to handle the AppRole auth to get a token, and then get the secrets it needs.
Is this not how it’s intended to be used?
What is the right way to leverage AppRole in k8s without saving vault credentials as a k8s secret?