Understanding Nomad reserved disk

Nomad
1

Hello,

I’m trying to confirm how the Nomad scheduler calculates disk resources.

On a physical host with a 10GB total disk partition, 8GB is used by the OS and other sutff, leaving only 2GB of actual free space.

However, Nomad successfully scheduled a job requesting a 4GB ephemeral_disk.

On my understand, this should not be the case. It seems that Nomad’s scheduler only looks at the total capacity of the disk (10GB in my case), completely ignoring actual available free space (2GB).

Is my understanding of this behavior correct? Is this working as designed, or is there a setting I missed to make Nomad aware of actual available disk space?

Thanks!

2

Yes, you’re not wrong. Nomad’s scheduler looks at the reported disk capacity of a node, not the free space the OS currently has. On a bare‑metal machine Nomad asks the node agent for the total size of the disk and uses that number when evaluating the ephemeral_disk requirement of a job. Because the default reserved_disk is 0, Nomad thinks the whole 10 GB is available, so a 4 GB request can be scheduled even though only 2 GB is actually free.

How to align Nomad with real free space

  1. Set client.reserved.disk on the client config – This tells Nomad to subtract a fixed amount of disk from the node’s capacity. Add this to the client config file and restart the agent.
client {
  enabled = true
  reserved {
    disk = 1024 # Megabytes to reserve
  }
}
  1. Use the disk resource in your job – The ephemeral_disk field already expresses the amount of disk a task needs. With reserved_disk in place, Nomad will not place jobs that would exceed the real free space.

With my understanding, the Nomad agent reports raw disk capacity so that the scheduler can make placement decisions quickly. The agent does not dynamically scan the OS for free space because that would add overhead. Therefore it is the user’s responsibility to reserve space that the OS or other non‑Nomad workloads will use.