Consul leadership selection in cluster when consul servers are down

Consul
1

Hi,
We have cluster of 3 consul servers (also -bootstrap-expect = 3). We run in production version 1.4.4
Now we were thinking to use 1.7.1 so we execute few case when consul servers were restarted or shut down. The next scenario is problematic for us, even though it is rare. It is not happening in 1.4.4

Lets say we have Servers: A (Leader), B , C

  1. Stop A --> Servers: B(Leader), C
  2. Stop B --> Servers C
  3. Start A --> Servers A,C

These 2 servers didn’t succeed to select leader of the cluster back , since B was last leader and it is down.

In the case that 3) Start B --> Servers B,C and B keeps to be leader.

Is this functionality that was changed recently? If we have 3 servers in cluster and 2 of the up they should be able to select leader or only to keep leader?

Thank you

2 Likes
2

I have found this explanation, but I am not sure if it’ll answer your question: https://github.com/hashicorp/consul/issues/454#issuecomment-64018111

When you follow the thread there will be some more interesting explanations.

3

You could add a node first (so you have a total of 4), then migrate the original 3 one at a time.

1 Like
4

Thank you for the answers.
I was talking about Not Migration situation but when we have Outage, so 2 of 3 Consul servers are Down and then 1 is back. Rare scenario, but still.

So I double checked my self again in 1.4.4 it is working and in 1.7.1 it is not. But I didn’t find anything looks related in Changelog…
According to Links above until there are 3 up again, due to split-brain we cannot get Leadership 100% only with 3. Am i right?

But it is working with older version…