Consul Connect CA w/ Vault

Hello,

I’ve had a consul cluster operational for around a month now that is using consul connect ca setup with vault. Everything has been working well, but in the past few days I’ve experienced 2 cluster down type of events.

The first one was related to the vault token used in the consul connect ca configuration, it was created as a periodic vault token, but expired the other day, so I had to replace it and update consul config with a new one. I may have mistakenly been under the impression it’d automatically renew itself as it was used by consul.

The second issue encountered was around the cert that consul had generated in vault pki for us based on the consul connect ca config, it expired and we began getting "cannot satisfy request, as TTL would result in notAfter ‘’ that is beyond the expiration of the CA certificate at ‘’ ".

Is it by design that these 2 issues need to be handled manually by us periodically as part of our internal process or should I expect these things to renew and rotate with vault automatically from consul connect ca?

I’m kind of fearful increasing the TTL or doing anything on the vault pki mounts that consul connect ca created automatically for us due to me bricking the first consul cluster because I meddled with it on the vault side instead of through consul.

So far, it seems like I’ll need to manually perform a ‘consul connect ca set-config’ and have it setup new ca/intermediary pki mounts in vault for the 2nd issue I encountered?

Thanks!

1 Like

I’ve also hit both of these issues and would love to see more documentation around how to handle these scenarios.

1 Like

I agree that the documentation can be better around connect and setup and how handle certificates.

About the Vault token, you can use Vault agent to keep the token renewed or use a systemd service and timer to keep renewing your token.

With regards to certificates, Connect supposed to handle them automatically. The CA is valid for 1 year and leaf certs 72hrs by default.

Thanks for the response, it’s helpful. That makes sense I can get the token periodically refreshing with vault agent or by writing a scheduled task/job to renew it, but this assumes that we have vault agent installed on a machine (assuming a consul server node). If we are already setting the periodic vault token into a consul config for connect ca to use, then to me, it makes sense that consul connect ca could allow and handle refreshing the periodic token for folks out of the box, and that was my initial assumption and expectation setting this up. I follow the intent there I think, Hashicorp gave us enough pieces to automate that one with a method we see fit, and doesn’t do it for us, that’ll work, but could be more clear where they draw the line with Connect CA.

Consul Connect CA handles the leaf certs and everything fine in day to day operations in the cluster that I can tell, but it is pretty wonky when creating or rotating the root CA and intermediary CA mounts in vault and it is not rotating it automatically. I don’t think we’re using defaults, so when the expiration time comes it fails and I’ve been having to manually perform a ‘consul connect ca set-config’ and make it rotate CA/Intermediary, so it’s either bad config somewhere, bugs, or the intent of how this is supposed to work I missed. If the intent is this should automatically happen, then I either have a bad config somewhere, missing policies or am hitting bugs.

I really enjoy the product suite and how flexible all the pieces are with it, but I feel the intent of some things I missed in the documentation, probably due to how complex some of the scenarios are that these products are solving.

2 Likes

This is old thread, but for someone reached here from searching consul connect vault… :wink:

Now consul >= 1.8.5 renews the vault token for consul connect.

In Consul 1.8.5 and later, if the token has the renewable flag set, Consul will attempt to renew its lease periodically after half the duration has expired.