Hi there,
Vault version: 1.4.0 (Docker)
I’ve setup auto unseal as per the guide here (https://learn.hashicorp.com/vault/operations/autounseal-transit)
I’m a little hazy on how this works under the hood, but I “think” the token I’m using to talk to the ‘upstream’ vault (step 2.2 in the guide) is being auto-renewed, but the renewed tokens are only stored in memory and are lost if I restart the vault.
I pass this token into the vault via /etc/sysconfig/vault as an environment variable at start time (because the guide strongly recommends against putting it into the vault config file).
The thing is, if I restart the downstream vault after a month of it being up, it won’t restart (with the following error):
Error parsing Seal configuration: Error making API request.
URL: PUT https://upstream_server:8200/v1/transit/encrypt/downstream_server
Code: 403. Errors:
* permission denied
The token I’m providing via the environment has expired (because it was generated > 1 month ago).
My question is, how do I set up a seamless method of remote unseal, where I don’t have to take manual action to generate new unseal tokens every month? (how come this important consideration is this not covered in the tutorial?). If the downstream vault is successfully renewing its token, is there a way I can get it to spit out the renewed tokens so that I can put them back into the /etc/sysconfig/vault file?
Many thanks,
Nick
2 Likes
The documentation here is not detailed enough about which tokens can be used specifically for CIDR, but this would be very helpful for this purpose. Maybe a developer can shed some light on this ?!
The tokens max ttl of 32 days is the system default, btw., and can be changed:
The system max TTL, which is 32 days but can be changed in Vault’s configuration file.
When no specific TTL was provided with the vault token create command, the generated token will inherit the default TTL which is 2764800 seconds (32 days). The same for the maximum TTL.
To tune the default TTLs:
Thanks for your input @Wolfsrudel,
I tried setting the token TTL > 32 days, but that’s the maximum any token (other than the root token) can ever be, without changing the Vault configuration. I’d rather not modify the this hard limit as I imagine pushing that to an unlimited value is strongly recommended against.
The only other way to go past the TTL limit is through token renewal and I “think” my downstream vault is renewing its token. However, the renewed token only seems to be stored in memory because when I restart Vault, I get the error shown in my post.
What I would like to know is how can I get Vault remote unseal to seamlessly renew its token so that I never have to take manual action to make sure my downstream Vault will start. I’m prepared to somehow grab the renewed token out of Vault and put it back into /etc/sysconfig/vault, but I’d like some guidance on how to pull the renewed token out of the downstream Vault.
Surely everybody that uses the transit secrets engine for remote unseal faces this exact same issue??
Many thanks,
Nick
Hi Nick,
I’m using periodic tokens https://www.vaultproject.io/docs/concepts/tokens#periodic-tokens.
The idea behind periodic tokens is that it is easy for systems and services to perform an action relatively frequently – for instance, every two hours, or even every five minutes. Therefore, as long as a system is actively renewing this token – in other words, as long as the system is alive – the system is allowed to keep using the token and any associated leases. However, if the system stops renewing within this period (for instance, if it was shut down), the token will expire relatively quickly. It is good practice to keep this period as short as possible, and generally speaking it is not useful for humans to be given periodic tokens.
Thanks @Vinclame,
Will Vault automatically attempt to renew ‘periodic tokens’ when using them in this manor (for remote transit engine unseal)? When they are renewed, does their value change? If so, how would I get Vault to export this new value so that I can feed it back in when I restart Vault (via the VAULT_TOKEN environment variable)?
Cheers,
Nick
Hi Nick,
Vault will renew the token but doesn’t change the value. Only the ttl is changed. For example creating a token with a period means the renewal has to take place within that period. The active vault server will renew the token from time to time. When the vault server is offline and therefore renewal stops, the token is revoked after the period is expired. Choose the period wisely because it depends on how long you expect the active Vault server is down.
You can test this yourself:
vault token create -period=5m -renewable=true
VAULT_TOKEN= vault token lookup
Now verify the TTL. This is set to 5m. Everytime you renew the is configured back to 5m. When you don’t renew your token anymore the token is revoked after 5m.
As described in the documentation " At issue time, the TTL of a periodic token will be equal to the configured period. At every renewal time, the TTL will be reset back to this configured period, and as long as the token is successfully renewed within each of these periods of time, it will never expire. Outside of root tokens, it is currently the only way for a token in Vault to have an unlimited lifetime."
1 Like
@nick-george did you get the solution for this, even m at the same situation.
please let me know if you find some solution.
Hi @Vinclame,
Thanks very much for your response. I’ll give it a crack and let you know how I go!
Cheers,
Nick
Hi @sgoogal,
I haven’t had a chance to give it a go yet. Will let you know when I do.
Thanks,
Nick
Hi,
I have created some periodic tokens, but have given them a long period (1wk or 168h), this is so I can have the downstream vaults down for some time without fear of them losing touch with the upstream vault server. I used this guide https://learn.hashicorp.com/vault/security/tokens#periodic-service-tokens.
I’ll report back when I’ve found out whether or not it worked.
1 Like
Hi,
I created a periodic token. However, if I restart the downstream/client vault after the initial expiration period, I get the following error:
vault[1632]: Error parsing Seal configuration: Error making API request.
vault[1632]: URL: PUT https://vault:58200/v1/transit/encrypt/clienthost
vault[1632]: Code: 403. Errors:
vault[1632]: * permission denied
systemd[1]: vault.service: main process exited, code=exited, status=1/FAILURE
It seems as though the original token does not last forever even though the client vault seems to be continually renewing its token… Not sure what is going on.
Thanks,
Nick
2 Likes
Hello, how would one generate a new unseal token once it’s expired? Since the unseal token is expired my GUI is down. I do not have the root token and only know my personal token when I’m able to log in via the GUI and view it. So I know I need to generate a new token but am unable to do so because I don’t have a token. I’ve tried to log in user auth method userpass but that still requires a token.
Hi there, not sure what you can do without the root token. I do the following on my Vault instance that holds the keys for another vault:
[root@boxybox ~]# export VAULT_TOKEN=ZZZZZZZZZZZZZZZZ
[root@boxybox ~]# vault token create -role=<hostname>
Key Value
--- -----
token XXXXXXXXXXXXXXXXXXXXX
token_accessor YYYYYYYYYYYYYYYYYYYYYY
token_duration 168h
token_renewable true
token_policies ["default" "<hostname>"]
identity_policies []
policies ["default" "<hostname"]
Then copy that token over to the host needing the auto-unseal.
It has been a while, so apologies if none of this makes sense.
Cheers,
Nick