Configure consul with systemd and environmental file

Consul
1

Hello,

I’m trying to set up consul with systemd, but I’m running into some issues. Running it from the cli works fine though. This is how systemd is configured:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target

[Service]
User=consul
Group=consul
EnvironmentFile=/etc/consul.d/consul.env
ExecStart=/usr/bin/consul agent -config-dir=/etc/consul.d/
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

/etc/consul.d/consul.env:

{
  "data_dir": "/tmp/consul",
  "log_level": "INFO",
  "server": true,
  "bootstrap-expect": 1,
  "bind": "10.16.16.68"
}

The error is:

Dec 14 15:07:49 consul-test-1 consul[2539]: 2021-12-14T15:07:49.496Z [WARN]  agent.router.manager: No servers available
Dec 14 15:07:49 consul-test-1 consul[2539]: 2021-12-14T15:07:49.499Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No known Consul servers"

Am I guessing the issue stems from the fact that consul isn’t reading the environmental json file?

If I run it from the command line, as follows:

consul agent -server -bootstrap-expect=1 -data-dir=/tmp/consul -node=consul-test-1 -bind=10.16.16.68 -enable-script-checks=true -config-dir=/etc/consul.d

I get:

2021-12-14T15:08:43.343Z [WARN]  agent.server.serf.wan: serf: Failed to re-join any previously known node
2021-12-14T15:08:43.343Z [WARN]  agent.server.serf.lan: serf: Failed to re-join any previously known node
2021-12-14T15:08:43.343Z [INFO]  agent.server: Adding LAN server: server="consul-test-1 (Addr: tcp/10.16.16.68:8300) (DC: dc1)"
2021-12-14T15:08:43.344Z [INFO]  agent.server: Handled event for server in area: event=member-join server=consul-test-1.dc1 area=wan
2021-12-14T15:08:43.344Z [INFO]  agent: Started DNS server: address=127.0.0.1:8600 network=tcp
2021-12-14T15:08:43.345Z [INFO]  agent: Starting server: address=127.0.0.1:8500 network=tcp protocol=http
2021-12-14T15:08:43.345Z [WARN]  agent: DEPRECATED Backwards compatibility with pre-1.9 metrics enabled. These metrics will be removed in a future version of Consul. Set `telemetry { disable_compat_1.9 = true }` to disable them.
2021-12-14T15:08:43.345Z [INFO]  agent: started state syncer
2021-12-14T15:08:43.345Z [INFO]  agent: Consul agent running!
2021-12-14T15:08:43.346Z [WARN]  agent: grpc: addrConn.createTransport failed to connect to {dc1-10.16.16.68:8300 0 consul-test-1.dc1 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.16.16.68:0->10.16.16.68:8300: operation was canceled". Reconnecting...
2021-12-14T15:08:50.539Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"
2021-12-14T15:08:52.362Z [WARN]  agent.server.raft: heartbeat timeout reached, starting election: last-leader=
2021-12-14T15:08:52.363Z [INFO]  agent.server.raft: entering candidate state: node="Node at 10.16.16.68:8300 [Candidate]" term=3
2021-12-14T15:08:52.368Z [INFO]  agent.server.raft: election won: tally=1
2021-12-14T15:08:52.368Z [INFO]  agent.server.raft: entering leader state: leader="Node at 10.16.16.68:8300 [Leader]"
2021-12-14T15:08:52.373Z [INFO]  agent.server: cluster leadership acquired
2021-12-14T15:08:52.376Z [INFO]  agent.server: New leader elected: payload=consul-test-1
2021-12-14T15:08:52.378Z [INFO]  agent.leader: started routine: routine="federation state anti-entropy"
2021-12-14T15:08:52.378Z [INFO]  agent.leader: started routine: routine="federation state pruning"
2021-12-14T15:08:53.603Z [INFO]  agent: Synced node info

So it eventually works fine.

Any ideas are welcome :slight_smile:

2

I posted too soon.
I understand now that it didn’t make much sense to create a json file as an environmental variable and that the environmental variables are much narrower in scope tha the cli arguments/config file options (Commands | Consul by HashiCorp)
So I simply create a new file consul.json inside consul.d with adapted content and now it seems to work correctly for now :slight_smile:

{
  "data_dir": "/tmp/consul",
  "log_level": "INFO",
  "server": true,
  "bootstrap_expect": 1,
  "node_name": "consul-test-1",
  "bind_addr": "10.16.16.68"
}

(reference: Configuration | Consul by HashiCorp)