Using integrated storage for Vault HA setup, there is one single raft.db file in the raft storage backend e.g. /opt/vault/data/raft/raft.db. I noticed that this file keeps growing regardless of creation or deletion operations.
I can understand that changing any data or metadata will increase the DB size. But what if I delete a K/V v1 secret or destroy a K/V v2 secret version/metadata? Which means that secret is permanently deleted and I will never need to access it anymore. Is there a way to shrink or compact the vault.db to free up the space occupied by permanently removed data?
In addition, worth to mention that even if I overwrite an existing k/v v1 secret which does not have multiple versions, or an existing k/v v2 secret already exceeding it max_versions, the DB size keeps growing all the same.
With this, if we open a vault for company use, the vault.db will grow sky high. It can easily max out the disk capacity by one bad operation/automation, and we have no way to clean things up.
It’s very unlikely that your KV engine is going to eat up a major portion of your space.
This isn’t for the faint of heart but you can inspect the storage to see what’s using what. This was simpler in the past with Consul as your backend where you could inspect the snapshots offline but it’s a bit more complicated with IS.
I did some extensive reading and this seems not an easy or straight answer. I believe it is due to GitHub - etcd-io/bbolt: An embedded key/value database for Go. that the space for free pages will not be reclaimed to free up the disk space, but can be reused by later transactions.