Shutdown idle services

Hi all! I’m currently experimenting with moving an internal shared platform that hosts multiple PHP services isolated inside containers. This currently runs using a home-grown orchestration system that we’re looking at replacing. It looks somewhat like this:

The containers just run the service in question and boot up when a requests come in.

For my new setup I plan on using Consul, Nomad and Traefik for proxying/load-balancing, with it looking roughly like this:

More or less the same layout, just switching our homebrew orchestrator for Nomad. I haven’t included Consul in the above diagram but we plan on registering services via Nomad so Traefik picks up routing through there. We’ll run Consul clients on the Nomad Clients and run a 3 server “servers/leaders” setup for both Consul and Nomad (basically the suggested architecture for both).

One thing I’m struggling to wrap my head around with Nomad is how to shut down idle services.

Our current workflow looks something like this:

  1. Request comes in for service1.bussiness.com
  2. If a container is running for it already, route request to it
  3. If a container isn’t running, start a container
  4. If we’re tight on resources, shutdown the least used contianer

I can mostly wrap my head around getting this into Nomad land and I believe it will look something like this:

  1. Request comes in for service1.bussiness.com
  2. If a Job/Service is running for it already, route request to it
  3. If a Job/Service isn’t running, register new Job with Nomad and wait for it to start

What I’m unsure how to do is if the Nomad cluster is low on resources how do I shutdown the least used Job to free resources on the cluster? We can horizontally scale to an extent but since we have heavy caching in front of these web apps there will likely be a large number of Jobs registered and running that aren’t actively being used that can be freed to start other services as they come in. Shutting down idle services would be our preferred course of action and what we currently do now.

Is this something thats commonly done with Nomad or is continually horizontally scaling more the done thing? One idea I had was using metrics from Traefik to somehow identify the least used service and stop it but I’m unsure whether there is a better/more common way!

I admit I’m very early on in my journey so feel free to tear this apart or point out if I’m doing something completely wrong. Hopefully this all makes sense and thanks in advance!

Hi @MikeTaylor,

I think the Nomad Autoscaler is what you need. That’s the github link, and here are the docs and some Learn Guides.

Hope that helps!

Derek and the Nomad Team

Hey @DerekStrickland! Thanks for the reply.

If I’m reading the docs right we might be able to set the max instances to zero hoping it scales back if requests are at zero for some time.

I’m not 100% certain whether this solves our problem since we only want to shut down idle services when resources are high though, as opposed to just always stopping all idle services. :thinking:

Depending on your use-case, you can consider a few other options as well:

  1. Running parameterized jobs (if your jobs are idleing, maybe you can run on demand instead)
  2. Set different priority & enable preemption.