Hi,
I am planning add HashiCorp Vault Token secret detection in Trufflehog, which is a opensource secret scanner. Specifically, I am interested in finding out the regex pattern for HashiCorp Vault tokens.
After reviewing the documentation, I was able to find the prefix pattern for the token. However, I am still unsure about the total number of characters in the token. where I can find this information?
Thank you in advance for your assistance.
I think it’s deliberately undocumented to accomodate potential future changes.
It has already changed a couple of times in Vault history.
A long long time ago (like, back in the 0.x days) tokens were just UUIDs.
At some point they changed to have the s., b., r. prefixes documented on the page you linked to, followed by 24 base62 characters.
But, if you’re using Vault Enterprise at these versions, and logging in to child namespaces, they get an additional . and 5 more base62 characters of namespace ID added on the end.
Then Vault 1.10 came along and changed things again - the token prefixes gained an additional hv, and all tokens except root tokens, got switched to a new format, in which the prefix is the same, but now everything after the prefix is base64 URL-safe variant without padding characters, encoding marshalled protobuf data. The changelog suggests the total length of such tokens will always be >= 95.
There is an environment variable feature flag, VAULT_DISABLE_SERVER_SIDE_CONSISTENT_TOKENS, to revert Vault 1.10 to the older token format.
But wait, there’s more… it’s deprecated and probably mostly forgotten, but still technically possible to create your own user-specified tokens as custom values!
For a newly implemented secret scanner, if you think you can assume most people will be using Vault 1.10 and not using VAULT_DISABLE_SERVER_SIDE_CONSISTENT_TOKENS, meaning you’d mostly be looking for
hv[sb]\.(?:[A-Za-z0-9]{24}|[A-Za-z0-9_-]{91,})
I left out recovery-mode tokens as they are inherently only useful during an interactive recovery session, so overwhelmingly unlikely to be committed whilst still valid.
Max is correct on all points here. I’d just like to add that VAULT_DISABLE_SERVER_SIDE_CONSISTENT_TOKENS is undocumented for a reason: it was intended as an escape hatch, because the change was a bit risky and we wanted a feature flag that could be used to revert it without downgrading if we discovered major problems after releasing it. Because it’s undocumented, we could remove support for it at any point without notice.