Vault Raft Stuck in Standby Mode

I have a cluster of 3 Vault nodes with integrated storage. Recently 2 of the nodes are down, and the remaining one node is in standby node.

What can I do to get new nodes to join the cluster? or is there anyway to restore the data to a new cluster?

Help would be much appreciated!

Running “vault status” shows that the “HA Cluster” and “Active Node Address” is pointing to an IP which doesn’t exist anymore. Is there any way to tell Vault to ignore that and make the only node left the leader?

Hello Boon,

As far as I understand you have a 3 node Raft cluster, and two of those nodes are down?

Are those node going to ever be joined back in the cluster?

If the answer is no, that would mean that your cluster has lost its quorum (in three nodes cluster the quorum is two) and you need to perform recovery procedure, by creating the peers.json file inside your Raft storage directory, and reduce the quorum to one, also include the remaining working node configuration in the json file. Here is a simple example of how to achieve that.