"Can't request Vault token for terminal allocation" when parsing template in poststop task


Faced strange problem. Have Nomad job with several tasks. One of the task I’d like to execute after main task stops. So lifecycle is set for “poststop”. But it cannot be executed because of strange error “Vault: server failed to derive vault token: Can’t request Vault token for terminal allocation”. And this happens only when task is marked as poststop. If I change nothing but lifecycle to “prestart” everything works perfectly.
What additional information should I provide to get any help? :slight_smile: Am spending too much time on this problem and cannot find what is wrong.

Thanks in advance.

I’m having the same problem. Did you find a solution?

I’m having the same issue as well - I’m wanting to do some final cleanup + backup and w/o vault available cannot get the keys.

I also realized that even before I had a template that used the value - the post-stop action reported a failure if I had the vault dependency declared at the group level.

… interestingly… with the template declared - it tried to render the secrets despite not getting a vault token and failed there… so looks like the system tried to go through the remaining work-items to create the task, even though it wouldn’t have run because the vault token wouldn’t have been issued.

Relevant code lines:

(update: dropped the inline code paste - didn’t realize github code links were rendered inline some)

… makes me think that there needs to be either:

  • special case handling for post-stop tasks (however in this context - it is run to get tokens for all tasks … so would be very messy)
  • pre-fetch tokens for post-stop tasks (while keeping them alive) so that when the post-stop task comes around, the token is ready