Creating HCL file using Python

Hi There, Just a question about the job HCL system. Sorry if it’s obvious.

I wanted to create HCL job files via python. Is there a simple way to do this; rather then editing a text file.

Hi @ravi,

AFAIK there’s no Python support for creating HCL files. There are libraries to read HCL1/HCL2 files into Python, but that’s about it.

The whole language is implemented in Go:

@ravi typically, HCL is targeted towards operators writing configuration by hand. If you are generating Nomad jobs, you might prefer using the JSON job specification instead.

There are some objects that take a slightly unexpected shape when you change them to JSON. It’s very useful to use a handwritten job that mirrors what you would like to make dynamic and run it through the Job Parse API or use the nomad job run --output «jobfile» command to have Nomad perform the transformation for you once.

There is also a third-party Python library for Nomad that might help reduce the amount of effort you have to programmatically make Nomad jobs.

1 Like

@angrycub Thanks I’ll try the manual process first and move into the JSON job specification. From the json job specification it says “This guide covers the JSON syntax for submitting jobs to Nomad.”

Does this mean I can use

nomad job run -output my-job.nomad

to output JSON version and use that to submit a job to nomad?

@ravi, you sure can. JSON jobs have to be submitted using the Jobs API. For example, you can use the sample job created by nomad job init to test this out.

nomad job init -short
nomad job run -output example.nomad > job.json
curl -XPUT $NOMAD_ADDR/v1/jobs --data @job.json

You can also fetch the JSON jobspec for a job already running in the cluster in a few ways

The Read Job API returns a Job object without the outer wrapping {"Job": ...}. You can use jq to add it after the fact like:

$ curl $NOMAD_ADDR/v1/job/example | jq '{"Job": . }'

The nomad job inspect -json «job-name» command returns the exact same information as the Read Job API. As an aside, the -json flag is a nice shortcut to fetching Nomad API output using the nomad CLI command (instead of having to remember curl syntax) for subcommands that it is available on.

The nomad job inspect «job-name» command returns a properly formatted JSON job suitable for resubmission to the cluster by adding the {"Job": ...} before outputting.

One last note, the JSON provided by the cluster for a dead or stopped job needs to be have "Stop" set to false before sending it back to the cluster.

Given this state

$ nomad status        
ID        Type     Priority  Status          Submit Date
example   service  50        dead (stopped)  2021-05-18T09:10:08-04:00
nomad job inspect example | tee job.json

If you scroll towards the top of the output you should note that "Stop": true. If you deploy this JSON as is, the new job will be created in the stopped state.

You can edit the file or use a tool like jq to mutate the .Job.Stop value on the way to a curl command that resubmits the stopped job.

cat job.json | jq '.Job.Stop=false' | curl -s -XPUT $NOMAD_ADDR/v1/jobs -d @-

Hopefully this gives you some extra context and food for thought.
-cv

3 Likes