Is it possible to automatically detect and reissue revoked secrets with consul-template?

Consul
1

I have a docker container running image hashicorp/consul-template:alpine with the following configuration:

vault {                                                                                                             
      renew_token = false                                                                                               
      vault_agent_token_file = "/home/vault/.vault-token"                                                               
      retry {                                                                                                           
        backoff = "1s"                                                                                                  
      }                                                                                                                 
    }                                                                                                                   
    template {                                                                                                          
      destination = "/etc/secrets/index.html"                                                                           
      contents = <<EOH                                                                                                  
      {{ secret "secret/my-secret/config" | toJSON }}                                                      
      EOH                                                                                                               
    }

It works well, the secrets get updated in realtime, all seems good.

But as I was playing around with revoking secrets, I noticed that the revoked secret doesn’t get detected by consul-template.

It seems like the consul-template, during regular operation, waits for 1/3 of the LeaseDuration of the secret before “calling home” again to refresh or reissue the secret (https://github.com/hashicorp/consul-template/blob/2734e5125874e6debb60765cf0e2ef1a65ee7737/dependency/vault_common.go#L155)

But is there any configuration option to tell consul-template to check perhaps once every 5 minutes for a revoked secret, while also respecting the LeaseDuration for renewal?

Maybe what I’m asking doesn’t make real sense, I’m aware that I do not fully understand all the concepts behind Vault.

In my mind, this is what should happen:

  1. consul-agent gets the secret for the first time, and keep the LeaseDuration somewhere

  2. every N minutes (customizable in the .hcl file), consul-agent checks if secret was revoked. If not, do nothing

  3. After 1/3 (or perhaps something else, like 1/2) of the LeaseDuration has passed, consul-agent should ask for a new secret. For the remainder of the 1/3 (or 1/2) of the LeaseDuration (transition between secrets), the two secrets should be simultaneously valid. This allows a transition time in which processes which depend on the rotated secrets can simultaneously work without any downtime. On the long run, using the 1/3 of the LeaseDuration as a cue to rotation, I think that it should result that, at any arbitrary moment, there are 3 simulatenous valid secrets (yet not revoked, one with 66.6 to 100% remainder ttl, another with 33.3 to 66.6% ttl remaining, and the last one with 0 to 33.3% ttl remaining).

Could someone please tell me the differences between what I think should happen (steps 1 to 3 above) and what in reality is implemented on consul-template?

And also, if it is possible to detect revoked secrets with consul-template?

Thanks everyone for any help provided.

2

Hey Rafael,

First let me couch this with the fact that I work on consul-template and have limited experience with Vault. I’ve mostly only played with the KV backend. So take this with a grain of salt.

When using the KV backend it is basically just a secure database for storing the secrets. It is up to you to handle situations suck as revocation. You could use multiple entries + lease duration to implement a rotation scheme, but it is up to you. It really depends on your usage pattern for the secrets how to best handle revocation.

Regarding your 1-3, for non-renewable secrets you have the basic flow down. Though it depends on your underlying process that CT is managing as to whether it accepts multiple keys during rotation.

I might be able to help more with more specifics, but hopefully this helped at least a bit.