Error on installing a agent

jas01 · March 8, 2022, 10:06am

Hi,

I’m trying to install a consul server and join a consul agent.

I follow the documentation Deployment Guide | Consul - HashiCorp Learn and currently it seem the server are running correctly (I’m guessing)

root@consul-p1:~# consul members
Node       Address              Status  Type    Build   Protocol  DC     Partition  Segment
consul-p1  x.y.z.52:8301  alive   server  1.11.4  2         paris  default    <all>
consul-p2  x.y.z.57:8301  alive   server  1.11.4  2         paris  default    <all>
consul-p3  x.y.z.59:8301  alive   server  1.11.4  2         paris  default    <all>

Now I try to add a agent and (auto-join) and it’s seem weird because event I got

root@consul-p1:~# consul members
Node       Address              Status  Type    Build   Protocol  DC     Partition  Segment
consul-p1  x.y.z.52:8301  alive   server  1.11.4  2         paris  default    <all>
consul-p2  x.y.z.57:8301  alive   server  1.11.4  2         paris  default    <all>
consul-p3  x.y.z.59:8301  alive   server  1.11.4  2         paris  default    <all>
consul-test  x.y.z.93:8301  alive   client  1.11.4  2         paris  default    <default>

I also got a lot of errors

root@consul-test:/var/log# consul info
Error querying agent: Get "http://127.0.0.1:8500/v1/agent/self": dial tcp 127.0.0.1:8500: connect: connection refused
root@consul-test:/var/log# systemctl status consul
                                                           
● consul.service - "HashiCorp Consul - A service mesh solution"
     Loaded: loaded (/etc/systemd/system/consul.service; disabled; vendor preset: enabled)
     Active: active (running) since Tue 2022-03-08 08:58:12 CET; 2h 6min ago
       Docs: https://www.consul.io/
   Main PID: 27291 (consul)
      Tasks: 93 (limit: 14745)
     Memory: 119.8M
     CGroup: /system.slice/libvirtd.service/system.slice/consul.service

Mar 08 11:04:26 consul-test consul[27291]: 2022-03-08T11:04:26.323+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..52:8300 error="rpc error making call: EOF"
Mar 08 11:04:26 consul-test consul[27291]: 2022-03-08T11:04:26.600+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..59:8300 error="rpc error making call: EOF"
Mar 08 11:04:26 consul-test consul[27291]: 2022-03-08T11:04:26.873+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..57:8300 error="rpc error making call: EOF"
Mar 08 11:04:27 consul-test consul[27291]: 2022-03-08T11:04:27.135+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..52:8300 error="rpc error making call: EOF"
Mar 08 11:04:27 consul-test consul[27291]: 2022-03-08T11:04:27.526+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..59:8300 error="rpc error making call: EOF"
Mar 08 11:04:27 consul-test consul[27291]: 2022-03-08T11:04:27.624+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..57:8300 error="rpc error making call: EOF"
Mar 08 11:04:27 consul-test consul[27291]: 2022-03-08T11:04:27.793+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..52:8300 error="rpc error making call: EOF"
Mar 08 11:04:28 consul-test consul[27291]: 2022-03-08T11:04:28.180+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..59:8300 error="rpc error making call: EOF"
Mar 08 11:04:28 consul-test consul[27291]: 2022-03-08T11:04:28.530+0100 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=x.y.z..57:8300 error="rpc error making call: EOF"
Mar 08 11:04:28 consul-test consul[27291]: 2022-03-08T11:04:28.530+0100 [ERROR] agent.anti_entropy: failed to sync remote state: error="rpc error making call: EOF"

I also trying to use

root@consul-test:/etc/consul.d# export CONSUL_HTTP_ADDR="x.y.z.93:8500"

which solve

root@consul-test:/etc/consul.d# consul info
agent:
        check_monitors = 0
        check_ttls = 0
        checks = 0
        services = 0
build:
        prerelease = 
        revision = 944e8ce6
        version = 1.11.4
consul:
        acl = disabled
        known_servers = 3
        server = false
runtime:
        arch = amd64
        cpu_count = 96
        goroutines = 45
        max_procs = 96
        os = linux
        version = go1.17.5
serf_lan:
        coordinate_resets = 0
        encrypted = true
        event_queue = 0
        event_time = 2
        failed = 0
        health_score = 0
        intent_queue = 0
        left = 0
        member_time = 15
        members = 4
        query_queue = 0
        query_time = 1

But still not working

root@consul-test:/etc/consul.d# consul kv get redis/config/connections
Error querying Consul agent: Unexpected response code: 500 (rpc error making call: EOF)
root@consul-test:/etc/consul.d#

Any idea where I’m doing wrong ?

Regards

jas01 · March 8, 2022, 11:09am

So I’m answering to myself just in case someone end up with the same problem.

In fact I was wrong about the interpretation of the part of auto encryption Deployment Guide | Consul - HashiCorp Learn for this part use Secure Consul Agent Communication with TLS Encryption | Consul - HashiCorp Learn

In the first document it’s not very clear (at least for me) the difference between the configuration on the server and on the agent. The second documentation make it perfectly clear