Don't restart Jobs when Vault Task Token TTL expires

Nomad
1

Hi,

it seems to me that services which rely on vault secrets are currently re-rendered and restarted when the TTL expires. Is there a way to not restart them?

2

Hi @maaft,

You can control this using the template stanza change mode job specification parameter.

Thanks,
jrasell and the Nomad team

3

Thanks!

Just to clarify: Will a job be restarted if the template rerenders, but the result is the same as before?

4

Hi @maaft,

It depends on the value of the change_mode within the template block, and that parameter only affects the task, not the entire job.

If you set the value to noop, the template will be re-rendered with the new data but no restart or signal will be sent to the task. Using signal allows you to configure change_signal so your application can handle re-reading the rendered template typically via SIGHUP, however, this is dependent on your own application code being able to handle this.

Thanks,
jrasell and the Nomad team

5

Oh, I know that it depends on change_mode.

The thing is, I see some kind of “diffing” in the nomad logs. Therefore, the question if the job also will be restarted if the rendered template is the same (of course given that change_mode is configured to do so). IMHO it makes no sense to restart a job, when the job description (incl. rendered templates) does not change. Or am I missing something?

6

@maaft,

I see what you’re asking now, thanks for the clarification.

The difficulty with rendered templates currently is that they are opaque to Nomad clients. It does look like it is possible to view the rendered template within the events Nomad receives from the backing library and process consul-template. This event, however, is fired after the re-rendered template has been written to disk. Nomad is only responsible for actioning changes as a result of a template event alongside setting up the template runner when a task is started.

I think it’s worth raising a feature request against Nomad for this so we can track, discuss, and gather further community interest. Whether it is technically possible to implement this without huge re-writes is another question.

Thanks,
jrasell and the Nomad team

1 Like