Hi, we have a use case where we need to build a job management framework using Nomad. Currently, we use Supervisor to manage our jobs and so that requires us to create a .conf file and putting it inside /etc/conf.d.
We are switching to Nomad due to the fact that It manages and provisions resources efficiently and automatically. With supervisor, we had to manually allocate the machines on which our jobs should run.
However, I’m stuck in an issue with Nomad such that whenever my job stops, I’m unable to retrieve its logs from the allocation directory. Do we have any mechanism to persist logs on some other folder like /var/log on ubuntu and configure log rotation there itself?
Ideally, I want to look up the logs after my job is stopped/restarted. On Linux, I would just go inside /var/log and see all the log files for my job being generated named job.out.log, job.err.log.2 etc.
Can I implement custom logs directory with log rotation so that my logs persists even after my jobs are stopped/restarted without using garbage collection interval?
Thanks a lot for your help folks!