Can I regenerate Vault recovery keys after KMS auto-unseal

I’m running a OSS Vault server with Consul backend on a kubernetes cluster. We are using AWS KMS keys for auto-unseal. It’s possible that the “recovery keys” created when the Vault server was launched were never recorded. We have a root token, but not the recovery keys.

Is it possible to regenerate recovery keys while I have the Vault server unsealed with my KMS key? (I’m afraid the answer to this is probably “no.”)

Reason: I’m trying to do replicate a Vault server to another AWS region and account to anticipate failure of AWS region and specifically the KMS key that it’s married to. My approach is to do a backup/key migration to Shamir keys such that I can create my DR Vault server elsewhere and start it /without/ auto-unseal using those Shamir keys.

I’m very much struggling with a way to replicate or preserve Vault data if the KMS key is unavailable. All of the mechanisms I’ve found (rekeying with custom KMS key, converting to shamir keys, etc.) involve disabling KMS temporarily and using recovery keys, which I don’t currently have.

Rich

Indeed you are not able to generate new recovery keys from Vault in your situation because the rekey process requires the recovery keys for security reasons. Since you don’t have them, rekeying vault isn’t an option for you, unfortunately. The way to solve this situation is to extract your secrets from Vault and then manually move them to a new Vault cluster. That’s not an easy process, Vault doesn’t have an option to mass export secrets, so that process can be tedious unless you write a script for this.

There is a third party tool to help dump recovery keys from memory, you can find the tool here: GitHub - bruj0/vault-recovery-key: This tool will decrypt your Vault recovery keys when using KMS. Per the documentation for that tool it only supports GCP and Azure KMS, not AWS KMS, so this doesn’t seem like a viable option for you unfortunately. I’m providing it to you so you can review it and determine if it’s something you want to try independently.

A similar tool which fetches the recovery key from Consul and decrypts it with AWS KMS is here: GitHub - cucxabong/hashicorp-vault-utils: Misc stuffs for fun.

It seems to have had AWS KMS support added last month.

1 Like