Vault pod restart time is increasing with increase in vault.db size

Vault
1

Hi all,

We have setup vault using helm.

We are using vault’s transit engine for encrypting data in our database. We have created 15,000,000 keys in transit engine.

The DB size is increasing continuously and reached 35GB.

We are facing an issue where where vault pod restart is taking more than an hour. Can someone give us any pointers on how to resolve this issue.

1 Like
2

This GitHub issue comment may give you some ideas about what may be happening: https://github.com/hashicorp/vault/issues/14635#issuecomment-1075387112

I guess not many people try to scale Vault to this size.

3

Thanks for the update @maxb

4

@maxb I want to set VAULT_RAFT_FREELIST=1 in env-variable. I have added following config:

extraEnvironmentVars:
  VAULT_CLIENT_TIMEOUT: 300s
  VAULT_RAFT_FREELIST_SYNC: 1

When the pod is restarted, I am not seeing this in env variables.

Please suggest how to enable VAULT_RAFT_FREELIST env variable.

5

Nevermind. It worked. There was an issue with syncing the config.

But the pod restart time after adding above config is still ~3 hours for us.

We tried adding config in following:

 extraEnvironmentVars: {}

server.postStart

The changes are reflecting when we add it in extraEnvironmentVars, But the load time is still not reducing.

Are we adding the config correctly?

6

Do bear in mind that the option enables saving information to disk to speed up future startups so will show no improvement for the first startup of each node, with the option set.

1 Like
7

Thanks for the help @maxb .
It worked on 2nd restart.

Our pod restart was taking nearly 3 hours, so I was a bit hesitant to restart it right away.