We are using 2 Vault server and 3 consul server as a storage.
We are doing rolling upgrade then rollback. Whenever we are doing rolling back on vault, vault login is failing with below error:
Error authenticating: error looking up token: Error making API request.
URL: GET http://127.0.0.1:8200/v1/auth/token/lookup-self
Code: 500. Errors:
- local node not active but active cluster node not found
On vault, we are getting below log:
2021-05-01T09:29:25.777+0530 [INFO] core: entering standby mode
2021-05-01T09:29:25.789+0530 [ERROR] core: failed to acquire lock: error=“failed to create session: Unexpected response code: 500 (rpc error making call: rpc error making call: Missing check ‘serfHealth’ registration)”
consul upgraded version: v1.8.3
consul rollback version: v1.6.1
vault upgraded version: v1.5.2
vault rollback version: v1.3.0
There is no acl enabled on vault or consul server.
vault config:
“listener”: [{
“tcp”: {
“address”: “x.x.x.x:8200”,
“tls_disable”: false,
“tls_cert_file”:“completepath_vault.crt”,
“tls_key_file”:“completepath_vault.key”
},
“tcp”: {
“address”: “127.0.0.1:8200”,
“tls_disable”: true
}
}],
“api_addr”: “https://x.x.x.x:8200”,
“cluster_addr”: “https://x.x.x.x:8201”,
“ui”: true,
“log_level”:“DEBUG”,
“storage”: [{
“consul”: {
“address”: “127.0.0.1:8500”,
“path”: “vault”,
“disable_registration” : “false”,
“service”: “vault”
}
}]
}
Post rollback, Vault init and vault unseal is fine but vault login by using root-token is failing.
Note: consul server version is 1.8.3 and vault consul agent version is 1.6.1