Hi,
We are dealing with unexpected behavior in scaling our applications. The scaling process starts right after the deployment is set to “health” because we don’t have a WarmUp or any other field we can set on Nomad to wait before scaling. At least I couldn’t find if I am wrong, please point me to the documentation.
When the app starts this answers with a status OK to the service check. This should not be a problem if we don’t have to warm up our apps(load some images base data to improve user experience) and when we run the warmup this consumes CPU/MEM which triggers the scaling on the warmup phase.
To avoid this behavior, we had to implement a custom service checker with an “internal warmup” before marking the service as healthy. So on our internal service we consider this warmup value and only starts answer with status OK after warmup.
Bellow is the service block that holds the warmup function.
“Services”: [
{
“Name”: “my-application-nomad-watcher”,
“TaskName”: “nomad-watcher”,
“PortLabel”: “watcher”,
“AddressMode”: “auto”,
“Address”: “”,
“EnableTagOverride”: false,
“Tags”: null,
“CanaryTags”: null,
“Checks”: [
{
“Name”: “health_check”,
“Type”: “http”,
“Path”: “/health”,
“Interval”: 5000000000,
“Timeout”: 4000000000,
“TaskName”: “nomad-watcher”,
“OnUpdate”: “require_healthy”
}
],
“Namespace”: “my-namespace”,
“OnUpdate”: “require_healthy”,
“Provider”: “nomad”
}
],