I have a process that dispatches a parameterized job and then wants to read the dispatched job’s result (which is written as a file in the alloc
directory). The dispatch
API call returns the job ID so I can use the API to read allocation IDs and then use the client
API to check if the result file was produced.
I’m trying to optimize this to not have to poll the API (check in a loop if the file was created). It looks possible but “hacky”:
- at the dispatched job’s start, create an empty file
wating_for_result
- the dispatching job calls “stream file” for
waiting_for_result
Client - HTTP API | Nomad by HashiCorp - when the dispatched job produces
result_file
, it deleteswaiting_for_result
- the dispatching job receives the event
file deleted
forwaiting_for_result
- the dispatched job can now read
result_file
I also thought about using the /event/stream
API to wait for the job/alloc finish but then I can’t be sure the result file will be available (Ensure alloc directories stay for some time - #3 by aartur).
So the question is if there are better ways to pass data between the jobs. The above solution might work but also looks not that easy to implement correctly (e.g. there can be race conditions between the dispatching job’s streaming initialization and the dispatched job result calculation).