Dead nomad job not purged by GC (Garbage Collection)

Nomad
1

I have a dead job in the Nomad UI that is never removed.


sudo nomad job status heartbeat
ID            = heartbeat
Name          = heartbeat
Submit Date   = 2021-11-02T20:47:03Z
Type          = service
Priority      = 50
Datacenters   = lon1
Namespace     = default
Status        = dead
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
nginx       0       0         0        4       0         0

Allocations
No allocations placed

nomad system gc doesn’t remove it.

How can I remove this job?
Other jobs dead jobs are removed as expected.

The cluster is up-to-date
Nomad v1.2.3 (a79efc8422082c4790046c3f5ad92c542592a54f)

1 Like
2

Hi @campbell. There are a number of rules which dictate whether a job is eligible for garbage collection. Have you tried running nomad job stop --purge heartbeat? This command will stop and force a GC of the job.

Thanks,
jrasell and the Nomad team

3

Thank you.
This removed the job.

# nomad job stop --purge heartbeat
==> 2022-01-03T13:32:14Z: Monitoring evaluation "b828332f"
    2022-01-03T13:32:14Z: Evaluation triggered by job "heartbeat"
==> 2022-01-03T13:32:15Z: Monitoring evaluation "b828332f"
    2022-01-03T13:32:15Z: Evaluation status changed: "pending" -> "complete"
==> 2022-01-03T13:32:15Z: Evaluation "b828332f" finished with status "complete"
1 Like
4

@jrasell any link to documentation or source code, the list of rules that affect the dead job GC?
we faced similar issue, and didn’t want to do manually purge each time the nomad server hit high memory

1 Like