Potentially sensitive data is printed to the vault logs

Vault
1

We noticed a case where potentially sensitive or secret data is being printed out the Vault logs in the case where the request is malformed. More specifically, if you issue a create/update request and one of the fields in this request is from an incorrect type (for example string instead of a map or vice-versa), the parsing of the request fails (as it should), and the log line that is associated with this error contained the actual content of the field.

This means that if a user makes a mistake while creating (or updating) a secret, their secret may be revealed to anyone who has access to the vault logs.

Here is a simple example that demonstrates this:
If you try to send a request to KV v2 Create or Update Secret API with this body:

{"data": "SOMETHING-VERY-SECRET"}

(data should be a map, but I deliberately made a mistake and put in a string with the secret value)

You get this line in the log:

2022-01-03T12:59:38.900+0200 [ERROR] core: failed to run existence check: error="error converting input SOMETHING-VERY-SECRET for field \"data\": '' expected a map, got 'string'"

Our actual case is more complex, as instead of KV, we are using custom engines that we develop ourselves, but the issue seems to be the same. We first noted this behavior in Vault 1.3, but it looks like it is the same also in 1.9.2.

Is this expected, and if so, is there any way we can avoid this? Is there some configuration option, or some way to mark some fields or paths in a way that will prevent their field values from getting to the logs when the request is malformed?

2

Audit logs are considered sensitive and should be treated as such.

3

So if I understand you correctly, it is considered fine by vault that secret values will be present in the logs because the logs themselves are defined as sensitive?

Our scenario is this: We have a separation between the people who run the operational side of Vault (networking, deployment, config, developing custom engines, etc) and the users who are actually storing and using the actual secrets. The team that operates the Vault needs access to the logs but should not have access to any user secret values. The specific case I described is an example where this separation breaks which is problematic for us.

What would be the best course of action in our case?

4

Operationally that makes sense, but sadly there is no way to lock out an admin completely. What we have are alerts and external loggers (Splunk) that capture and track the usage. If there is a question then we can trace the access back to when/who did what.

5

Might be worth opening an issue for this. This seems like that data could be hashed by default, which would obviously make the admins’ jobs harder but certainly less chance of data leakage.