What all does the allocation Modified column encapsulate

Nomad
1

I have the following allocation; a raw_exec java process.


Allocations
ID        Node ID   Task Group               Version  Desired  Status   Created     Modified
42056ea3  ebdbda91  my-pulsar-broker         25       run      running  1mo24d ago  1mo24d ago

When I updated the Nomad (and Consul) binaries in place and restarted the systemd services for Consul and Nomad, it changes to the following:

Allocations
ID        Node ID   Task Group               Version  Desired  Status   Created     Modified
42056ea3  ebdbda91  my-pulsar-broker         25       run      running  1mo24d ago  3m34s ago

Note: I updated the binary versions as follows:


Consul :  1.12.2  ->  1.14.4
Nomad  :  1.3.1   ->  1.4.4

The Pulsar (java) process kept running fine during the external agent upgrade; I confirmed with ps output.

I had the following questions:

  1. What change(s) causes the Modified column to change as the process is running fine?

  2. In case the allocation process does restart, for any reason (during the agent version upgrade) I would see the same “external” change in the Modified column. (an alloc status would reveal more)
    2.1. Is there a way to easily understand if the task process restarted, without having to look at the allocation status ?

2

:bump:

Any pointers to existing documentation will also be fine. :confused:

3

Hi @shantanugadgil!

  1. That Modified column comes from the allocation’s ModifyTime. This gets updated whenever we need to make a write to raft for that allocation. The primary place you’ll see this happen is when the client updates the status of the allocation. Usually that’s during a deployment or when new task events happen (mounting a volume, Docker container exited, etc.). But when the client “restores” its connection to a running allocation on restart, it needs to check back into the server with the current status of the allocation as well.

  2. I don’t think so. Something to keep in mind when you’re looking at the job status summary view is that allocations can have more than one task, so all the information about the allocation is necessarily “compressed” into a high-level ClientStatus field (which is the Status column on the job status command). Getting into per-task behavior is really only visible by looking at the individual allocation via alloc status.

1 Like