How could I query a Nomad agent for ports reserved dynamically?

Nomad
1

Hello,

I’m writing some playbooks in Ansible to automate the deployment for a number of jobs. What I’d like to do is, after starting the Nomad job, query the agent for which ports have been assigned to each task and use that data to write rules for UFW. I can run nomad job inspect <jobname>, but it’s not clear to me from the documentation which field I can reliably get the info I need from. I’m also not familiar at all with Go templating and whether I can massage the data into a JSON format like [{"taskName: "web", "port": "9000"}, ...]. My inclination would be to write a Python script, but I’d rather not if I don’t have to.

Any help in getting and formatting the port info would be appreciated!

1 Like
3

Hi @sleepy.tent0930 :wave:

nomad job inspect will give you the jobspec information that you submitted, so it won’t have any runtime data, like the dynamically allocated ports.

What you want to do instead is to look for the allocation details. You can request this using the -json flags, but unfortunately it won’t give you the allocation resource information.

Once you get to this point of programmatic automation it might be best using the Nomad API directly.

For example, you can use the /v1/allocations endpoint with resources=true as a query parameter. From there you can jq the result:

$ curl -s 'http://localhost:4646/v1/allocations?resources=true' | jq '.[].AllocatedResources.Shared.Ports'
[
  {
    "HostIP": "127.0.0.1",
    "Label": "http",
    "To": 23588,
    "Value": 23588
  }
]
[
  {
    "HostIP": "127.0.0.1",
    "Label": "http",
    "To": 0,
    "Value": 8080
  },
  {
    "HostIP": "127.0.0.1",
    "Label": "api",
    "To": 0,
    "Value": 8081
  }
]
[
  {
    "HostIP": "127.0.0.1",
    "Label": "http",
    "To": 28814,
    "Value": 28814
  }
]
[
  {
    "HostIP": "127.0.0.1",
    "Label": "http",
    "To": 25191,
    "Value": 25191
  }
]

If you do find needing something more complex, and want to write a Python script, there’s a good community client available:

We are also working on official OpenAPI clients as well:

1 Like
4

That’s super helpful, thanks! When I run this query, does it just return allocations for the single host machine (localhost, in this case), or does it include everything in the entire datacenter?

5

Yeah, my example was kind of bad because I only had a local dev agent, but that endpoint will give you all allocations in your cluster :+1:

6

Is there a way to restrict it to the host I query? Not all hosts in my cluster are running the same tasks. I’m writing an Ansible module to handle this, so practically speaking I can just filter those allocations where HostIP == ansible_default_ipv4.address, but this wouldn’t work if (theoretically, not me) someone had Consul bound to a different address, right?