Constraints for Waypoint server on Nomad cluster


Waypoint seems to work perfectly. Although there is a problem I cannot solve: I’d like to run Waypoint server and runner on a certain node in a Nomad cluster. But I don’t see any possibility to do that. They become installed anywhere Waypoint wants. And if node fails those are redeployed on another node and stop working because runner cannot access server or server changes it’s token. Or something like that.
There could be great to have possibility to provide some constrains to Waypoint while installing server on Nomad. This would allow fixing Waypoint services onto a certain nodes in a cluster.

Hi @vadim.cesonis!

Yes, this is a known limitation in how we waypoint install sets up Nomad. This is due there being no standard load balancer within Nomad.

Constraints are an interesting way to solve this problem. We could easily just add the ability to pass constraints in from the command line when setting up the Waypoint server.

But as you pointed out, if that node fails then Waypoint won’t be redeployed if the constraint fixes it to a certain node only. So I fear we’d have the same issue as we currently do.

I’ll talk with the Nomad team about the best way to handle this as well.

Hi Evan.

Thank You for the reply.

While thinking about these constraints I was trying to solve a little bit another problem: I just want to manage cluster resources so that “system” nodes host all the staff we need to run as an additional services to our own. There can be some queue managers (like RabbitMQ), maybe smtp service, etc. They just have their own resource pool and use it without affecting main microservices we need to run. To ensure stable Waypoint working there can be several nodes where server and runner can be installed.

Would like to repeat my main idea: I just want to have more controllable cluster. :slight_smile: I have nodes where I run my staff and I want them to be flexible: I should be able to start, stop or restart them on demand. It’s normal when some system requirements are changing and I have to update Nomad’s configuration file (for example to add a volume). If then I restart Nomad I want to be sure that CD pipeline is still working.
In our system we have nodes where changes can be done quite frequently. And also there are nodes, which are allocated for a single service only, so there no changes will be applied for a very long time. I think that it could really be great to have possibility to manage where Waypoint instances should run. Nomad does have all these features. :slight_smile:

As You mentioned, there could be great to have an additional optional parameter in Waypoint command setting constraints (attribute in meta data for example) that should be referred to when deploying Waypoint server and runner instances.