Operator pattern in Nomad

Nomad
1

I would like to run AWX in Nomad. Unfortunately, they have chosen a very k8s-centric approach with how they distribute their software. They have a docker-only thing for development only, but they seem to insist on running an operator pattern for their production deployments, and when looking at what it does, it seems to use (among other things) k8s secrets in their code, which seems to make a straight-up conversion to run in nomad even harder.

From what I can tell, Nomad doesn’t have a direct match to the operator pattern, and from at least one conference talk about it, it seems to be for good reasons. Are there any resources (or people with good ideas, I guess) when it comes to adapting and moving something from a k8s/operator-centric approach to a more orchestrator-neutral way of doing things, and what difficulties one might expect to stumble over and possible solutions or workarounds?

1 Like
2

Could you please provide a link to this, I’m interested.

I’ve only seen this example of the operator pattern being used in Nomad:

Are you familiar with it?

Clearly it’s a long way off from something like the operator SDK , but I would love to exchange opinions on this.

3

Hi @brucellino1 !

So I was trying to figure out how to tackle the operator problem watching Stop Writing Operators and I thought it was a really good talk on the problem of overusing the operator pattern and maybe applying it in ways that aren’t really adding value, but, admittedly, this doesn’t as much solve the problem as it is making me even more frustrated about how it’s being used.

As far as I am concerned, this is not so much about the operator per se, but rather finding a way to use applications where the dev team have decided that K8s is the only orchestration technology available. Do you have a different usecase than I do? Do you have the need for writing your own within Nomad? What are you trying to solve?

I have glanced at that article just when starting to evaluate Nomad at work, just to understand what and how you would work with that sort of approach, but as I mentioned I was hoping to find a way to convert an existing one to work with nomad, rather than rolling my own, so it didn’t seem to solve my problem at the time, even though I should give it a closer look, now that I know a bit more about what I can do in Nomad I might understand it a bit better, so thanks for the link! Have you written something using the article? Are you happy with it? What would you wish worked better?

4

I agree :100: with the sentiment. My feeling on the subject is that these are not “applications using the operator pattern”, but “applications bundled with some internal management features for K8s”. It’s no different than packaging something for Linux as opposed to for MacOS. Folks have long argued that K8s is the new Operating System, and I’m ok with that, but it’s not really a pattern if it’s specific to one particular piece of software, IMHO.

My feeling is that without an SDK similar to the operator framework, we will have to write our own code to implement them. As in the K8s situation, this will have to be delivered via Nomad natives (jobs, groups, tasks, services, etc), and packaged with nomad-pack.

I have only re-implemented what was done by the original author, in go. I can see where it’s going, and how to potentially do this in other languages, using the sdks. All in all, I can see how this can solve many problems, but I haven’t gone too deep in it.

One of the problems I want to address is for example progressive, automatic addition of task drivers on nodes, via the operator pattern. For example, starting with a bare-bones OS and Nomad client, the operator would install and configure a container runtime environment update nomad config to enable the driver (raw-exec task, restart nomad service), etc.

I am currently doing this via Ansible roles, so I could probably write a Nomad job to do this, but the key thing is that I want it done automatically when nodes join the cluster. It’s a different use case to “install a product”, like deploying, say MinIO Couchbase, etc for which operators exist and would need to be reworked in order to respect the business logic for Nomad.

1 Like
5

Thanks for your response, interesting to hear! I have too little understanding of how in my case AWX uses it’s operator, but it seems that the hard part would be to decipher what’s going on with their playbooks to replicate the functionality in Nomad. I am not familiar enough with the AWX codebase, but to me it seems rather opaque in how it’s constructed, and it’s not immediately clear how to rewrite the operator functionality in a Nomad-centric way.

I am not sure I understand exactly what your usecase is, or maybe I am not clear on the meaning of “task drivers” in this case, would you care to share a bit more about it? I am doing initial install in my cluster running on coreos at deployment time, but it seems that there might be additional configuration for me that might be more similar to your described usecase than the AWX deployment does. I am looking forward to hear more about your progress if you’re interested to share!