Hello there! I have faced with problem during joining new node to cluster. I have transit node for auto unseal and HA cluster consisting of several nodes, all nodes placed in AWS cloud, also nodes are unsealed and have Integrated storage (Raft) backend. Config used to deploy new nodes already had been used for deployed worked nodes. New node successfully find all other nodes in the cluster (by auto_join function) and try to get bootstrap challenge from master node. (according Vault documentation) But when a new node executes a PUT request to the master node, it receives “500 error” with the description “Error making API request”. So auto unseal is unavailable until node do not join to cluster.
Important to note, config haven’t been changed, the internet connection between a new node and other nodes is stable and works fine.
vault[8444]: error=
vault[8444]: | error during raft bootstrap init call: Error making API request.
vault[8444]: |
vault[8444]: | URL: PUT http://172.20.200.1:8100/v1/sys/storage/raft/bootstrap/challenge
vault[8444]: | Code: 500. Errors:
vault[8444]: |
vault[8444]: | * Error making API request.
vault[8444]: |
vault[8444]: | URL: PUT https://transit-vault-node.local-env.com:8200/v1/transit/encrypt/autounseal
vault[8444]: | Code: 403. Errors:
vault[8444]: |
vault[8444]: | * permission denied
Also i can see logs like these on leader node:
vault[3756]: URL: PUT https://transit-vault-node.local-env.com:8200/v1/transit/encrypt/autounseal
vault[3756]: Code: 403. Errors:
vault[3756]: * permission denied"
What could be the reason for this trouble?