Nomad leader not accept jobs

Nomad
1

Hello. We have a cluster with 6 servers with same nomad configuration and version, hardware parameters ( cpu,ram,disks) etc. All jobs are located on the nomad client servers, but only the leader not accept any jobs. What is the reason?

2

Hi @yyovchev can you describe your setup in a little more detail? As described in the operating guide, a Nomad agent runs as either a Client or a Server (or both). Clients run the actual tasks, and Servers control the Clients, handle job submission, etc. Are you sure Nomad is configured as a Client on every node you expect tasks to run?

3

Hello @seth.hoenig thank you very much for your fast reply.
I have6 servers - all servers are client and server both.
Network settings are:
host1: 172.16.0.21
host2: 172.16.0.22
host3: 172.16.0.23
host4: 172.16.0.24
host5: 172.16.0.25
host6: 172.16.0.26

The current leader is host5.
If I restart it, other server will be become a leader. For parameters “advertise” I set the correct IP for the current host (for example host1 I setup 172.16.0.21 for “advertise” parameter). For “retry_join” parameter I skip himself IP - for example for host5 I skip 172.16.0.25 (please see the configuration bellow).

Here is my configuration for host5 (the current leader):
cat /etc/nomad.d/nomad.hcl

data_dir = “/var/lib/nomad”
datacenter = “XXX”
bind_addr = “0.0.0.0”

advertise {
http = “172.16.0.25”
rpc = “172.16.0.25”
serf = “172.16.0.25”
}

server {
enabled = true
bootstrap_expect = 2
server_join {
retry_join = [“172.16.0.21:4648”,“172.16.0.22:4648”,“172.16.0.23:4648”,“172.16.0.24:4648”,“172.16.0.26:4648” ]
}
}

client {
enabled = true
network_speed = 1024
node_class = “hosting”

}

plugin “raw_exec” {
config {
enabled = true
}
}

consul {
address = “127.0.0.1:8500”
}

Can you help me about this issue? Thank you very much for your time!