Access nomad from within nomad

Nomad
1

I’m looking in to setting up a micro service that can create nomad jobs from within nomad. I can’t find any information of how one can set that up. Is there a minimal example of how to do it? Is it even possible?

Similar to how one can use docker if one mounts the docker socket in to the container, i.e. docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install docker.io -y ; bash", and rbacandClusterRole` in k8s to be able to control the cluster from inside the cluster.

If the cluster has a public ip one can of course call it on that url, but it would be nice not requiring a public ip such one can use it for local development etc.

1 Like
2

Have you tried Nomad’s HTTP API yet?

… or one of the SDKs?

And you definitively don’t want to expose such a service via public IP. :wink:

3

Hi @joh4n,

Thanks for using Nomad!

This has come up before. We’re currently exploring options for making this a first-class feature and where to prioritize it, but in the meantime, you could write your own job controller using the HTTP API. You should be able to inject the client IP using env { HOST_IP = “${attr.unique.network.ip-address}” } and then access it using the HTTP API.

If you are programming in go, you can check out the api package in the main Nomad repository for a fully baked API Client implementation. Also, we have a new experimental openapi project that is under active development. It doesn’t cover all endpoints yet, but it has Jobs, and it has multiple language targets. If you don’t see your language of choice you can even add your own. Take a look at the Makefile and the README for instructions and examples of how to generate clients for other languages. Also, feel free to submit a PR if you do add another language or add coverage for other endpoints. We’re definitely looking for contributors.

I hope that helps.

Cheers,

@DerekStrickland and the Nomad Team

1 Like
4

@DerekStrickland Thx that should work, I just could not figure out how to get that ip. I was thinking of using the HTTP API or Home - python-nomad for now, as I probably will do a quick setup of this in python for now.

Is there a way to monitor a nomad cluster for events, jobs completing/failing etc? I found GitHub - seatgeek/nomad-firehose: Firehose all nomad job, allocation, nodes and evaluations changes to rabbitmq, kinesis or stdout but it does not look to be under active development. Ideally using amqp such I can just consume it from rabbitmq and not have to ping and api every 10s.

@fhemberger The plan is to use the HTTP API or simmilar sdk. But my question is how to get that ip to call from within the cluster. That was the ip that could be public assuming one enables auth for nomad not the service itself. Sorry if that was unclear.

5

@joh4n

There is a way! Check out the Events API. Also, this project has some example code for how to use it.

  • Derek
6

Hi @joh4n, I’m the creator of python-nomad and also a maintainer. While it took us a bit to add event stream support. You can now in the 1.4.1 version create a listener against the event stream to send results to a more established message broker/pub sub. It’s not exactly like firehose but give you the capability to integrate into your stack upon your discretion.

With Hashicorp adding an official openapi specification I imagine this will be the way to to go to generate client libraries once it has full compatibility with all of Nomads routes. python-nomad also doesn’t have 100% route support as well, but hoping to bridge that gap till the openapi spec is.

7

The events api seems like exactly what I was looking for :slight_smile: I should probably read trough the https://www.nomadproject.io/api-doc I have just been reading on Documentation | Nomad by HashiCorp

8

I think Advertise Address is better suited here:

      env {
        // Ensure this interface is where Nomad advertises it's HTTP port.
        NOMAD_ADDR = "${attr.nomad.advertise.address}"
      }

I was running a Nomad agent in dev mode locally on localhost, so attr.unique.network.ip-address was coming as 127.0.0.1.

Running with -bind 0.0.0.0 or explicitly setting advertise address in config and reading that value should make the HTTP API accessible.