I’m considering a migration from EKS to Nomad for a number of reasons, but there’s one specific use case that I can’t find an obvious way of supporting. Bascially, we want to:
- Use a mixture of spot and ondemand autoscaling groups
- Launch batch jobs that will prefer to use spot instances
- If there is no spot capacity across all of the instance types in the ASG for a (preferably configurable) given period (e.g. 5 minutes) automatically fall back to using ondemand instances.
So basically, if there is spot capacity, use it, but use ondemand if there isn’t any for a timeout value.
In Kubernetes terms, this would be usually implemented using node affinity, though practically it mostly has to be implemented in the autoscaler there as well.
I’m not averse to building strategy/target plugins to implement this behaviour if it isn’t doable with the existing Nomad, but it’s not clear to me if this is even possible within the current plugin architecture.
Can anyone provide guidance/suggestions as to how to achieve this?