Tags flapping in server nodes

Consul
1

After sorting out the mess on my cluster as exposed on my previous post, I started migrating the consul servers to the latest, 1.11.2, while most of the clients in 0.8.5 and I am facing a really weird scenario: Tags are literally flapping in the new servers.
Flapping meaning they disappear and reappear completely, from all the 5 servers. They start in synchrony, and then one disappears, then another, then all, the all back… I couldn’t find a pattern, although when tags are back they don’t last more than 20-30 secs and off again for 1-2 minutes.

consul denitition couldn’t be simpler

    "service": {
        "port": 8300,
        "id": "consul",
        "name": "consul",
        "tags": ["cicely", "server", "staging"]
    }
}

Tags are not changed dynamically in our environment, and never on consul servers, on agents, when changed, it’s directly on the file followed by a consul reload.

Any ideas?

2

Ok… Am I right to assume that consul servers belong implicitly to the service “consul”? If I remove that definition, they boot up and work in the same fashion, and, in fact, if I change the service id/name to something different i.e.: konsul, there are two services per node (as in multiple service definitions), and, in those secondary ones, tags work as expected.

I assume this is by design, right?
If so, I have found my workaround already, by creating that secondary service with the tags I need.

Regards.

3

Yes, that is correct. The consul service is used to advertise the addresses of the Consul servers.

This is by design. Consul does not currently support the ability to set custom tags on the consul service. (See hashicorp/consul#2299).

Registering a second service with the desired tags is the correct alternative solution.