Hi,
I’d like to figure out “who” is making requests to Consul.
With the “consul.kvs.apply” metric we found that we are experiencing a lot more KV applies than before in certain datacenter. With the “consul.rpc.request” metric we found our Consul servers in that datacenter are getting more RPCs. Now, I want to know who exactly is making those extra requests.
I tried to look at “consul.client.rpc” grouped by host (host tag added by our host-local datadog agent), but I didn’t see any obvious host doing more RPCs than before. In our setup, multiple apps running on the same host uses the single consul client agent running on that host. So I’m guessing the app making those extra consul RPCs is multi-instance app running on multiple hosts, making the RPC count change per-host not prominent.
Is there any way I can track down who actually makes RPC requests, when they make requests through host-local consul client agents?
Any help is much appreciated. Thank you in advance.