We are getting these warning messages
dns: Skipping invalid node for "servernode1" NS records
dns: Skipping invalid node for "servernode2" NS records
dns: Skipping invalid node for "servernode3" NS records
dns: Skipping invalid node for "servernode4" NS records
dns: Skipping invalid node for "servernode5" NS records
servernode5. The first node and second node are working fine.
These messages suddenly come without any changes on the servers.
I tried to investigate a lot but I can’t find any useful information about this.
I am not sure what’s wrong with the node3,4,5 but this issue causes a metric spiked.
Please advise how to resolve this issue.
The error you are seeing should only be logged if the node name contains characters other than
-. (See consul/agent/dns.go.)
What version of Consul are you using? Can you also provide the output of
consul operator raft list peers so that I can see the names of all servers in your environment?
Hi @blake Thanks for the response.
We are running Consul in version 1.6.1.
This is the output of
consul operator raft list peers
sg-consul-6024.agprod1.agoda.local fbb78719-a534-af71-aebf-a4a0232e6a55 10.104.206.71:8300 follower true 3
sg-consul-6020.agprod1.agoda.local 8a3fea0f-fad6-686d-276f-265874e981be 10.104.206.72:8300 leader true 3
sg-consul-6021.agprod1.agoda.local 738edfec-ba08-d094-4ce7-a22a1e6146a7 10.104.205.35:8300 follower true 3
sg-consul-6022.agprod1.agoda.local e52238dd-80ad-b61f-028a-9f20e075711c 10.104.206.73:8300 follower true 3
sg-consul-6023.agprod1.agoda.local d31caefc-4350-5d88-ca2e-c3851a8bd73d 10.104.209.205:8300 follower true 3
The issue happens only on
sg-consul-6024. And from the raft list peers, the output is the same for all nodes.
The side effects we see are that those all issue nodes will call too many RPC calls.