Failed to join No installed keys could decrypt the message

Hi @wusikijeronii,

Did you by chance attempt to change the encryption key at some point? If so, the process for changing the encryption is slightly more involved.

When a Consul agent starts for the first time, it will read and attempt to use the encryption key defined in the encrypt parameter of the agent’s configuration file. After the initial startup, the agent will create and store the gossip key in the $DATA_DIR/consul/serf/local.keyring file.

At this point, if the agent is subsequently restarted, the gossip key will be read from the keyring file, not the encrypt parameter in the agent configuration. (This behavior can be changed by setting disable_keyring_file to true in the agent config.)

If you need to change/rotate the key, you can do so using the consul keyring command. This command allows for you to install a new key, and safely rotate all agents to the key without requiring each individual agent’s configuration be updated, or each agent be restarted for the new key to take effect.

A step-by-step tutorial for rotating gossip keys can be found here.

2 Likes