Procedure for Vault install with Helm and auto-unseal

Vault
1

I’m going to deploy Vault with raft backend on Kubernetes using Helm. My preference is to use an existing cluster using the transit engine for auto-unsealing. However, I’m a little confused on what the requirements are for this to be done efficiently. It appears I need to inject the unwrapped token from the transit cluster as an environment variable of VAULT_TOKEN. The configuration must point to the transit cluster as well. Finally, something needs to run a vault operator init when the first node comes up.

Here’s a snippet of my values file for the relevant pieces.

...
      postStart:
        - /bin/sh
        - -c
        - vault operator init
...
      extraSecretEnvironmentVars:
        - envName: VAULT_TOKEN
          secretName: unseal-vault-token
          secretKey: token
...            
      seal "transit" {
              address = "http://transitvault.example.com:8200"
              disable_renewal = "false"
              key_name = "autounseal"
              mount_path = "transit/"
              tls_skip_verify = "true"
            }

However, when I install the chart the pods continuously go into CrashLoopBackoff and are showing errors, that I haven’t been able to pull admittedly, related to the vault operator init command. I will try to get that command soon.

But are there recommendations on how to best do this since there are a few components which need to be setup properly to work. Thank you in advance.

2

This is a really bad idea … you’re telling the pod to initialize vault every time they start. Let the nodes start, they won’t go into ready but you can shell into them and init manually (which should only be done once).

3

I removed that and things appear to have improved. 4 of the 5 Vault nodes come up with one exception. Any idea what this means? I’m not sure why this node is different from the others.

2021-10-25T19:52:39.870Z [INFO]  core.autoseal: seal configuration missing, but cannot check old path as core is sealed: seal_type=recovery
2021-10-25T19:52:43.248Z [INFO]  core: stored unseal keys supported, attempting fetch
2021-10-25T19:52:43.248Z [WARN]  failed to unseal core: error="stored unseal keys are supported, but none were found"
2021-10-25T19:52:44.074Z [WARN]  core: join attempt failed: error="error during raft bootstrap init call: Put "http://vault-2.vault-internal:8200/v1/sys/storage/raft/bootstrap/challenge": dial tcp 192.168.65.232:8200: i/o timeout"
2021-10-25T19:52:44.074Z [INFO]  core: security barrier not initialized
2021-10-25T19:52:44.074Z [INFO]  core: attempting to join possible raft leader node: leader_addr=http://vault-3.vault-internal:8200
2021-10-25T19:52:44.104Z [WARN]  core: join attempt failed: error="failed to send answer to raft leader node: error bootstrapping cluster: cluster already has state"
2021-10-25T19:52:44.104Z [INFO]  core: security barrier not initialized
2021-10-25T19:52:44.104Z [INFO]  core: attempting to join possible raft leader node: leader_addr=http://vault-4.vault-internal:8200
2021-10-25T19:52:44.113Z [WARN]  core: join attempt failed: error="failed to send answer to raft leader node: error bootstrapping cluster: cluster already has state"
2021-10-25T19:52:44.113Z [ERROR] core: failed to retry join raft cluster: retry=2s
2021-10-25T19:52:44.856Z [INFO]  core: security barrier not initialized
2021-10-25T19:52:44.856Z [INFO]  core.autoseal: seal configuration missing, but cannot check old path as core is sealed: seal_type=recovery

Update: There was some old data laying around so I think that was causing problems. Thank you!