Greetings,
We are running our service in kubernetes pod with sidecard containing consul agent, also main consul cluster as statefulset.
When any casual problem occurs on service side - k8s reloading pods and service registers in consul without problems.
But we faced some floating problem - something happens (maybe network issues?), then service can’t reregister in consul and got errors:
rpc error making call: failed inserting node: Error while renaming Node ID: "4833aa15-8428-1d1a-46d8-9dba157dbc60": Node name xxx.xxx.xxx is reserved by node c5dc5b48-f105-79f0-7910-3de6629fddd0 with name xxx.xxx.xx
Also service returning 200 - OK healthcheck, but consul forbids registering.
Service:
- Spring Boot - 2.2.4
- Spring Cloud - Hoxton
Sidecar consul agent cli:
- consul
- agent
- -retry-join={{.Values.global.consul.server}}
- -datacenter={{ template “alv.consul.datacenter” . }}
- -data-dir=/consul/data
And ‘{“leave_on_terminate”: true}’
Duplicate Node IDs after upgrading to 1.2.3 · Issue #4741 · hashicorp/consul · GitHub but we are encountering same problem on fresh consul version - 1.9.0
