Hi. We have:
- 3 server running Nomad version 1.7.5 servers
- using Nomad version 1.8.0 from a docker container
- deploying nomad jobs as part of Jenkins CI/CD automatically
From time to time we are getting the following errors, which fail the pipeline:
+ nomad job run streamlit.nomad.hcl
==> 2024-07-02T15:39:17Z: Monitoring evaluation "304e26d6"
2024-07-02T15:39:17Z: Evaluation triggered by job "streamlit-lokilogs-kcukrowski-gtsavtamer-17456"
2024-07-02T15:39:19Z: Evaluation within deployment: "2f5f5ad5"
2024-07-02T15:39:19Z: Allocation "b4a9facc" created: node "492ea091", group "lokilogs"
2024-07-02T15:39:19Z: Evaluation status changed: "pending" -> "complete"
==> 2024-07-02T15:39:19Z: Evaluation "304e26d6" finished with status "complete"
==> 2024-07-02T15:39:19Z: Monitoring deployment "2f5f5ad5"
==> 2024-07-02T15:39:19Z: Error fetching deployment: Unexpected response code: 404 (deployment not found)
script returned exit code 1
How to fix the error? What could cause the error?
Does Nomad 1.8.0 command line tool opens a single network connection to the server? If there are 3 servers, can it be related that the “fetching deployment” connects to a different Nomad server then the deployment was submitted to, which the other server is missing the deployment?
Thank you.