After upgrade nomad to 1.3.1 deployment fails

Nomad
1

I upgraded my nomad agent version to 1.3.1 on one of the nodes of the cluster.

Every job deployment after that fails with port collision error:

+$ nomad job run -verbose -check-index 2318283 job.nomad
==> 2022-07-04T16:28:06+05:30: Monitoring evaluation "59c7211d-6159-a4e9-1e5a-7b2cb16dbc73"
    2022-07-04T16:28:06+05:30: Evaluation triggered by job "iptables-test"
    2022-07-04T16:28:06+05:30: Evaluation within deployment: "268048f3-40f4-2698-e76e-b5d363a412b3"
    2022-07-04T16:28:06+05:30: Evaluation status changed: "pending" -> "complete"
==> 2022-07-04T16:28:06+05:30: Evaluation "59c7211d-6159-a4e9-1e5a-7b2cb16dbc73" finished with status "complete" but failed to place all allocations:
    2022-07-04T16:28:06+05:30: Task Group "iptables-test" (failed to place 1 allocation):
      * Constraint "${attr.unique.hostname} == eu-pl-02": 1 nodes excluded by filter
      * Resources exhausted on 1 nodes
      * Dimension "network: reserved port collision http8=80" exhausted on 1 nodes
    2022-07-04T16:28:06+05:30: Evaluation "34fb2838-ec48-d4ed-8b50-1449f46b776f" waiting for additional capacity to place remainder

How to figure out why this error occur? How do I debug this? Any of you guys faced similar issue?

PS: I’m a noob at Nomad

2

Hi there.
I’m a noob too, but I think that is the problem:

Dimension “network: reserved port collision http8=80” exhausted on 1 nodes

The job is trying reserve port 80, but the port could be busy.

Maybe you need use dynamic ports: https://www.nomadproject.io/docs/job-specification/network#dynamic-ports

3

Hi, sorry for not mentioning this in the question. There is no job running on the node so the ports are free. I tried dynamic ports as well. Same result.

4

Could you post your job file?