Moved to new Node Group and now get permission denied with root token

Vault
1

We had to move all our PODs on our Kubernetes cluster to a new node group including the vault pods. We might have moved them before they were synched because now I cannot login with the root token. When I do a vault status from any of the vault PODs I see that it is still unsealed and initialized.

/ $ vault status
Key Value

Recovery Seal Type shamir
Initialized true
Sealed false
Total Recovery Shares 5
Threshold 3
Version 1.14.1
Build Date 2023-07-21T10:15:14Z
Storage Type raft
Cluster Name vault-cluster-xxxxxxx
Cluster ID -------
HA Enabled true
HA Cluster https://vault-1.vault-internal:8201
HA Mode active
Active Since 2023-08-22T18:10:22.654403637Z
Raft Committed Index 464
Raft Applied Index 464

I saw some error in the leader POD’s logs but then it seemed to have been part of the other PODs not being online and then returning. I honestly don’t know what to do to check since most vault commands won’t help without being able to login. Do I need to scrap it and start over or can I do anything to restore?

2

I figured out the issue and hopefully someone can use this to help. All we had to do was use the unseal keys to generate a new root token. Part of the learning curve but also good to know how and why to do this.