How to overcome the chicken and egg problem of consul and vault encryption?

For a full hashicorp stack, the guides and docs recommend that for production consul and vault deployments, they should talk to each other encrypted, and to benefit from the security of automatic key rotations, vault should be used as the certificate authority.

What is the recommended way to get consul, vault, and nomad securely communicating with each other, with the benefit of automatic key rotation, from initiation?

So far this question doesn’t have a simple answer, as the documentation avoids answering it for the sake of simplicity.


So far the options seem to be:

Without Automatic Key Rotation

Manually generating certs with terraform tls provider

  1. use terraform tls provider to generate the tls certs
  2. configure and boot consul, vault, nomad with the certs

Seems until v0.14 is out which supports a sensitive flag, this is insecure

Manually generating certs from Vault

  1. boot vault
    2 generate certs from vault
  2. reconfigure vault, and configure consul and nomad to use these certs
  3. reboot vault, and boot consul and nomad

Now that vault, nomad, and consul all support sans, the code here may now work

Manually generating certs from Consul

  1. boot consul
  2. generate certs from consul
  3. reconfigure consul, and configure vault and nomad to use these certs
  4. reboot consul, and boot vault and nomad

With Automatic Key Rotation

I just can’t seem to find instructions for this. All the instructions seem to be related to Consul Certificate Authority, which is only used automatically (with automatic key rotation) for Consul Connect, which Consul Connect is just used to secure the communication between apps, not the actual consul/vault/nomad services.

Hope

I really hope that there becomes a way to boot consul, nomad, and vault, with all of them just using either Consul or Vault as the certificate authority, and able to secure themselves automatically with TLS cert rotation.

What is essentially needed is fo rthe auto_encrypt flag to be extended to auto_encrypt not just the consul clients, but also the consul servers, as well as vault and nomad agents (clients+servers).

All the hashicorp products tout features that should make things extremely secure, yet the actual software doesn’t know how to use them.

Relevant Links