Call to sys/init API results in "failed to unseal barrier: Vault is not initialized" error

Vault
,
1

I have vault running in one pod (secretstore-cskm-0) and I’m making requests to initialize, unseal vault, after I confirm it’s installed, from another pod (secretstore-post-install-job-xxxxx). I’m using HA mode and using Mariadb-maxscale (secretstore-mariadb-maxscale-0) as backend.

So when we make this call to vault sys/init API - curl --insecure -g -s https://[2001:db8:1234::3a49]:8200/v1/sys/init

it returns - {“initialized”:false}

So now we go on to initialize vault -
curl --insecure -g -s --request PUT --data ‘{“secret_shares”: 1, “secret_threshold”: 1}’ https://[2001:db8:1234::3a49]:8200/v1/sys/init

and it returns - {“errors”:[“failed to unseal barrier: Vault is not initialized”]}.

What is the exact cause for this error?

I also have to mention that this issue is sporadic. Sometimes the PUT request to /sys/init API works fine.

2

Not sure whats going on, but might help if you can post the output of vault status, run from each node with VAULT_ADDR pointing at itself.

1 Like
3

Here’s the information you requested.

The first call from secretstore-post-install-job-xxxx pod to secretstore-cskm-0 pod results in

curl --insecure -g -s https://[2001:db8:1234::3a6e]:8200/v1/sys/init
OUTPUT: {“initialized”:false}

Now I make a PUT request to initialize vault:
curl --insecure -g -s --request PUT --data ‘{“secret_shares”: 1, “secret_threshold”: 1}’ https://[2001:db8:1234::3a6e]:8200/v1/sys/init

The first call results in : {“errors”:[“failed to unseal barrier: Vault is not initialized”]}

Now because it failed, I try to initialize it again -

curl --insecure -g -s --request PUT --data ‘{“secret_shares”: 1, “secret_threshold”: 1}’ https://[2001:db8:1234::3a6e]:8200/v1/sys/init

OUTPUT NOW: {“errors”:[“core: barrier reports initialized but no seal configuration found”]}

After that all the subsequent PUT requests result in the same error - core: barrier reports initialized but no seal configuration found

STATUS:

in secretstore-cskm-0:
bash-4.4$ curl --insecure -g -s https://[2001:db8:1234::3a6e]:8200/v1/sys/seal-status
{“type”:“shamir”,“initialized”:false,“sealed”:true,“t”:0,“n”:0,“progress”:0,“nonce”:“”,“version”:“”,“migration”:false,“recovery_seal”:false,“storage_type”:“mysql”}

in secretstore-post-install-job-xxxx
bash-4.2$ curl --insecure -g -s https://[2001:db8:1234::3a6e]:8200/v1/sys/seal-status
{“type”:“shamir”,“initialized”:false,“sealed”:true,“t”:0,“n”:0,“progress”:0,“nonce”:“”,“version”:“”,“migration”:false,“recovery_seal”:false,“storage_type”:“mysql”}

in secretstore-mariadb-0 (backend):
bash-4.2$ curl --insecure -g -s https://[2001:db8:1234::3a6e]:8200/v1/sys/seal-status
{“type”:“shamir”,“initialized”:false,“sealed”:true,“t”:0,“n”:0,“progress”:0,“nonce”:“”,“version”:“”,“migration”:false,“recovery_seal”:false,“storage_type”:“mysql”}

Like I mentioned before if I deploy the same helm umbrella chart again with no values changed, it might be successful or it might fail. This issue is Sporadic.

Please help me pinpoint the issue.

4

vault_stderr logs-

bash-4.4$ cat /opt/vault/logs/vault_stderr.log
2020-11-12T22:23:08.458Z [INFO] proxy environment: http_proxy= https_proxy= no_proxy=
2020-11-12T22:23:08.458Z [WARN] storage.mysql: No TLS specified, credentials will be sent in plaintext. To mute this warning add ‘plaintext_connection_allowed’ with a true value to your MySQL configuration in your config file.
2020-11-12T22:23:18.191Z [INFO] core: security barrier not initialized
2020-11-12T22:23:29.484Z [INFO] core: security barrier not initialized
2020-11-12T22:23:33.278Z [ERROR] core: no seal config found, can’t determine if legacy or new-style shamir
2020-11-12T22:23:33.282Z [INFO] core: security barrier not initialized
2020-11-12T22:23:33.393Z [INFO] core: security barrier initialized: stored=1 shares=1 threshold=1
2020-11-12T22:23:33.403Z [ERROR] core: failed to unseal barrier: error=“Vault is not initialized”
2020-11-12T22:25:29.169Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:25:43.263Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:25:43.263Z [ERROR] core: error checking health: error=“core: barrier reports initialized but no seal configuration found”
2020-11-12T22:25:56.965Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:25:58.371Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:25:58.371Z [ERROR] core: error checking health: error=“core: barrier reports initialized but no seal configuration found”
2020-11-12T22:26:13.468Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:26:13.468Z [ERROR] core: error checking health: error=“core: barrier reports initialized but no seal configuration found”
2020-11-12T22:26:25.603Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:26:28.566Z [INFO] core: seal configuration missing, not initialized
2020-11-12T22:26:28.566Z [ERROR] core: error checking health: error=“core: barrier reports initialized but no seal configuration found”
2020-11-12T22:26:43.660Z [INFO] core: seal configuration missing, not initialized

5

Hey Mike,

Let me know if you need any more information

6

If its sporadic, could be connection/network setup to your storage backend.
I would rule out mysql being the issue here (assuming its an empty db instance each time, and not being reused) and change storage to integrated storage, retest and see if that gets your further.
You might also post your config files if someone could spot something misconfigured.

7

Okay I will test further.

But what I want to know is - why that particular error? In what case is it thrown? I’m just initializing, I’m not unsealing and it says “failed to unseal barrier”?

8

You can try this.
This is happening because the default state of the vault is Sealed. You need to unseal the vault in order to resolve this issue and use the vault.

You need to generate unseal keys and then use them to unseal the vault.

Generate the keys:

kubectl exec -ti vault-0 -n vault -- vault operator init

Unseal the vault using the keys shared above until the threshold is met:

 $ kubectl exec -ti vault-0 -n vault -- vault operator unseal

 $ kubectl exec -ti vault-0 -n vault -- vault operator unseal

 $ kubectl exec -ti vault-0 -n vault -- vault operator unseal

On the prompt paste different keys each time.

You can get detailed steps here: Deploy HashiCorp Vault on Kubernetes using Helm | DwOps