Understanding when/why in-place updates occur

When a change is made within a job/group stanza (but not task), an in-place update is performed, and by the looks of it, the update stanza gets ignored, shown in the picture below in A when doing nomad job plan.

When a change is made within a task, then the update stanza is followed as shown in B (with 1 canary). The update stanza is honoured as expected.

I can’t figure this out from the documentation (unless I am reading the wrong parts). A few questions below…

  1. Why does this happen?
  2. What are “rules” to know when an in-place update will occur (besides running job plan)? Maybe that would be explained by the “why” above.
  3. Does “in-place” allocations risk introducing downtime as all the new allocations get allocated? In the UI it looks like all of them are re-allocated at once, ignoring the update stanza. If yes, what is the best way to mitigate downtime when a change has to be done outside the task stanza?
  4. This indicates that changes shouldn’t really be done to jobs/groups “on-the-fly” like code changes and expect 100% uptime, is this understanding correct?

Hello

What exact changes did you make to the job that you are asking about? Please post the output of diff. I do not fully understand. Please kindly post text instead of images of text. Please kindly post the code.

Why does this happen?

Why does what happen exactly? The meta values of the task are inherited from the group and the job meta values.

What are “rules” to know when an in-place update will occur (besides running job plan)?

From experience they are somewhat fuzzy and handled case by case. Bottom line, everything that requires to restart the process results a destructive update.

Removing meta key changes the environment variable NOMAD_META_AKEY of the process. To change an environment variable of the process it has to be restarted.

Does “in-place” allocations risk introducing downtime as all the new allocations get allocated?

I do not understand. There are no new allocations in an in-place update.

what is the best way to mitigate downtime when a change has to be done outside the task stanza?

https://developer.hashicorp.com/nomad/docs/job-declare/strategy/blue-green-canary

This indicates that changes shouldn’t really be done to jobs/groups “on-the-fly” like code changes and expect 100% uptime, is this understanding correct?

I also do not follow this question. To have 100% uptime deploy a new version, make sure it works, switch your proxy to the new version, make sure it works, shutdown the old version - i.e. have canary deployment. It is kind of agnostic to what code changes you make.