Cannot Get Tiny Cluster Working

Nomad
1

Hi,

Need some help with a nomad error.

Background

I’m trying to get a 2 node cluster working (the reason it is 2 node is because of the use case requirements).

I am simulating the 2 node environment using Vagrant. The 2 nodes are “n1” and “n2”. They are running ubuntu.

There are 2 ethernet interfaces on each node, eth0 and eth1. Both boxes default to the same ip address on eth0 (10.0.2.15). But eth1 is 172.20.20.10 on n1 and 172.20.20.11 on n2. So, I’m using eth1 for all communications.

Consul Setup

First, I bring up consul on n1: sudo consul agent -server -data-dir="/tmp/consul" -bind="172.20.20.10" -bootstrap-expect=2

Then, I bring up consul on n2: sudo consul agent -server -data-dir="/tmp/consul" -bind="172.20.20.11" -join="172.20.20.10" -bootstrap-expect=2

Consul appears to come up correctly.

Nomad Setup

I start nomad like this on n1: sudo nomad agent -server -data-dir=/tmp/nomad -bind=172.20.20.10 -bootstrap-expect=2

Then I start nomad on n2: nomad agent -client -data-dir="/tmp/nomad" -servers="172.20.20.10"

Output

The n1 nomad output:

    2021-04-11T19:22:18.338Z [ERROR] worker: failed to dequeue evaluation: error="No cluster leader"
    2021-04-11T19:22:27.935Z [ERROR] http: request failed: method=GET path=/v1/agent/health?type=server error="{"server":{"ok":false,"message":"No cluster leader"}}" code=500

Any ideas what the problem is or (probably) what I’m doing wrong?

Thanks
-T

2

while bootstrapping your server your declare “-bootstrap-expect=2” which means the server should expect to see at least 1 other server before an election can be held. Since node 2 is only a client the quorum is never reached an the leader election never happens.

So to fix you either need to make node n2 a server too, of reduce the bootstrap expectations

2 Likes