I am trying to install a watch on one node to check for certain changes in a service. I am getting this error on the masters as well as the following error on the agent:
2021-08-05T10:19:18.881Z [ERROR] agent: subscribe call failed: err="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection closed" topic=ServiceHealth key=rabbitmq-cluster-admin failure_count=12
Unfortunately i cannot find anything about this on google.
I ran into this when my consul servers were configured with
rpc {
enable_streaming = false
}
but my consul agents had
use_streaming_backend = true
the agents were trying to use streaming rpc instead of the traditional blocking queries, but my servers didn’t have rpc enabled.
in my case it’s because I didn’t follow the upgrade documentation: my servers were on 1.9.x (where rpc is disabled by default) and I tried running 1.10.x clients (where use_streaming_backend is enabled by default). presumably you could get into the same situation by manually setting the configs too, I guess.