Hello all,
I am new to Vault, and to secrets management in general.
I work on an application which keeps some sensitive data encrypted in a database. The encryption key is stored in a config file on disk. I’m investigating how to avoid having that encryption key stored on disk, specifically to address the threat model of someone gaining access to the server, and therefore the config file, and being able to use it to decrypt the sensitive data in the database.
At first glance it seems there are many ways Vault can help with this. For eg the simplest option I came across is the KV secrets engine which could store the encryption key. The application would retrieve it from Vault, instead of from disk. Alternatively the Transit secret engine could handle the encryption and decryption for me, and there’s no encryption key to store anywhere. These both sound great.
But no matter which option I look at, I keep coming back to the same problem: how does my application authenticate itself to Vault, without running into the same problem I am trying to address?
-
If I use tokens to authenticate, then I need a token saved on disk so my application can read it and ask Vault for the encryption key. This does not seem to be any improvement in the threat model I am trying to address - an attacker who gains access to the machine can find that token just the same as they could find the encryption key now. They just need to use the token, get the key, and I’m back to where I started. I don’t think adding this extra step makes any difference, any attacker skilled enough to have gained access to the machine won’t be slowed down by having to trace through my application to replicate what it does.
-
If I set up the application to automatically generate and renew tokens, then an attacker on the machine can do the same thing and generate themselves a new token. And anyway the tokens need to be stored so the app can use them.
-
If I use some form of AppRole, then my app needs a role ID and a secret ID to access Vault. So I need a role ID/secret ID saved to disk, which seems to defeat the purpose? As I understand it AppRoles can have extra restrictions like only valid for particular hosts/users. Neat, but again, if an account on the machine is authorised to access Vault, then an attacker with access to the machine can do it too.
-
All 3 community-built PHP SDKs listed on Vault’s libraries page simply have tokens hard-coded on disk. How is this any better than storing the secrets in a file on disk and not using Vault at all?
-
Consul Template needs a token from somewhere to access Vault, and then automatically generate config files with secrets on disk. Back to square one.
-
Envconsul - same as Consul Template but secrets are saved as environment variables. As I understand it an attacker with access to the machine can access those env vars much the same as they can access config files on disk. No improvement.
-
Vault Agent Templates much the same, though requires AWS or Kubernetes, I am using neither;
-
Platform Integration - I am not using any of the supported cloud infrastructures (AWS, GCP, Azure …).
-
Trusted Orchestrator - I am not using an orchestrator;
-
Vault Agent - AFAICT involves writing tokens to a sink, which is a file on disk. Attacker can access this too?
I feel like I am missing some basic principle, or misunderstanding something fundamental. This is all new to me so I may just not be grokking things properly yet.
What am I missing? How can Vault help in this scenario?
At the most basic secrets need to be handled by your application for it to achieve whatever the purpose of it is. If you are attacked by a highly skilled adversary who has gained full root/administrator privileges on a machine all bets are off - the attacker has access to read the system memory as well as all files on the machine, so could find any secrets currently being held. While you could encode those secrets in memory to try to delay the attacker there must be some mechanism to convert those encoded secrets into plaintext using the information held on the server to allow the application to run.
If you step down a level and assume the attacker is less skilled or doesn’t have the same level of access various techniques including the use of Vault can help.
At this level storing the secret in memory is preferable to storing it on disk. While file permissions can protect access to a file it can be easy to make mistakes and allow wider access than is desirable or accidentally store copies elsewhere due to the use of temporary files. Stopping the application would totally remove the secret from the server as it isn’t stored on disk - which is very useful in an emergency.
One way of ensuring the secret is never on disk is to fetch it from an external system such as Vault. As you say there is then the issue of how to authenticate your application. The different options offer different levels of trade-offs. Probably the most secure option would be to use a one time wrapped credential. That could be injected into the system at startup or entered automatically or manually. If it were injected at system startup an attacker could intercept the credential before the application. While that theft would result in the attacker obtaining a valid access credential it would also alert the application to the theft - as it is one time the application would fail to get the credential, alerting to possible interception.
This is where many of the advantages of Vault come in. While it is impossible to differentiate between the real application and an attacker if they have a sufficient level of access, all requests to Vault are logged using the audit capabilities. As a result you can see exactly what the attacker has been doing, and may be able to detect them if their usage patterns are different from the real application (e.g. an attacker might try to find all secrets and fetch them, which is less likely for the real application).
Additionally Vault supports the management of dynamic secrets. These are often used as part of the access control to other systems such as databases. Instead of using long-lived and usually broad credentials Vault is able to generate a unique credential for every individual application, with privileges tailored to their needs. The potential damage of an attacker is therefore limited, both by reducing what they can access as well as the duration they can do so (before having to talk to Vault again).
With a static database credential even if an attack is detected response can be difficult. Disabling access could easily mean that all other legitimate users are blocked requiring distribution of new secrets to them all. With dynamic secrets you could disable the attackers access specifically, with them having no way of returning.
Ultimately there is no way to prevent an attacker from impersonating a legitimate system given sufficient skill and access. However tools like Vault can increase the skill/access requirements for some forms of attack, but also importantly providing visibility of what the attacker is doing as well as limiting their long-term impact.
1 Like
Going back to your specific use case, with the decryption done by the application with the key stored locally an attacker could steal both the key and all the encrypted data. They can then decrypt at their leisure by which time you can’t do anything.
If instead you used the transit engine while the attacker could still steal the encrypted data they wouldn’t have the decryption key. They could call Vault to decrypt each entry (assuming they have stolen the Vault access credentials) but have to remain “on site” during that process. You would hopefully detect the unusual request pattern and then disable their access, meaning they have both been detected (previously they might not have been notices) and the damage limited (they only managed to decrypt some of the data - and you know exactly what they did manage).
1 Like
Thank you very much for your detailed replies. This helps a lot, and clarifies some of the big holes in my understanding.