I have a service that requires a DB to be up and running. As they are intrinsically related, I am trying to keep them together in the same job.
However, if I put them in the same group as separate tasks, I have observed the following:
- The db will start coming up
- The service will come up, and attempt to connect to the db
- This fails, and the service crashes
- The failure is noted by nomad, and the both tasks in the group are terminated to try again.
- Obviously, the second attempt has the same result, as the DB now must go through initialization once again.
So, I attempted to split it into two task groups. However, whichever group I place first will be allocated and executed, and apparently the job is waiting for that group to finish to do the next one. So even if I put the DB first, the service will never be allocated and thus never come up.
This seems like such a common use case, I assume I am missing something obvious. What is the recommended approach to do this in nomad?