Recommended Spread Method in a DC?

Nomad
1

Hi folks,

Just looking to crowdsource some thoughts here - we have a single DC with multiple workers in a nomad cluster, and we want to spread our jobs evenly across all workers. We’re working with an older version of nomad right now and after a restart, it seems to dump all jobs onto one worker, which isn’t great.

We don’t have any additional attributes or configuration at the minute. What would be the best method of having:

  • Spread across all available workers
  • Don’t dump them all onto the same worker
  • Don’t block deployments if all but one worker are unavailable

I don’t want to end up in a scenario where, should we lose all but one worker, it’ll refuse to deploy because we’ve set up very strict rules to prevent that - but I want it to avoid that scenario unless absolutely necessary. What would your thoughts be? How would you handle the spread, using which attributes?

Thank you for your thoughts!

(Edit: Based on this: Nomad doesn’t spread processes evenly on all servers - Nomad - HashiCorp Discuss is node.unique.id the best? What would happen in the scenario where there’s only one node remaining?)

2

Guess you want to change the SchedulerAlgorithm : Scheduler - Operator - HTTP API | Nomad by HashiCorp
The default is binpack, which makes a lot of sens when using public cloud resources, much less when you’re running on prem or with dedicated servers. In this case, you can set the scheduler to spread

3

That looks like…exactly what I need! Thank you!

An additional question on this if you don’t mind - our environment is already bootstrapped. If I update via the API, does that persist going forward even between cluster restarts, etc? Or will it return to the bootstrapped value at some point?

Thanks!

4

Hi @Eambo,

The update and configuration is persisted to state and therefore will survive server restarts and upgrades.

Thanks,
jrasell and the Nomad team

5

Thank you both kindly for the answers, this looks much healthier for our environment now and with minimal effort compared to changing our job templates :slight_smile: