The consul daemonsets constantly log the following errors and our services also return about hundreds of these but there is no impact to our services everything is running healthy.
consul 2022-06-28T00:51:23.670Z [ERROR] agent.client: RPC failed to server: method=Health.ServiceNodes server=<redacted_ip>:8300 error="rpc error making call: rpc error making call: EOF"
consul 2022-06-28T00:51:23.670Z [ERROR] agent.http: Request error: method=GET url=/v1/health/service/PHName?tag=version:0.0.11.11002&tag=environment:TEST&passing&dc=usw1&index=2662639&wait=300s from=<redacted_ip>:48926 error="rpc error making call: rpc error
So its more spammy more than anything.
consul info:
agent:
check_monitors = 0
check_ttls = 17
checks = 17
services = 17
build:
prerelease =
revision = ba7d9435
version = 1.8.2
consul:
acl = enabled
known_servers = 3
server = false
runtime:
arch = amd64
cpu_count = 16
goroutines = 406
max_procs = 16
os = linux
version = go1.14.6
serf_lan:
coordinate_resets = 0
encrypted = true
event_queue = 0
event_time = 4
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 40697
members = 37
query_queue = 0
query_time = 1