Hello,
I am trying to run consul-k8s with an existing external consul servers with acls enabled. values.yaml for the helm chart as follows:
global:
enabled: false
name: consul
domain: "external.dc1.com"
datacenter: "dc1"
gossipEncryption:
secretName: consul-gossip-encryption-key
secretKey: key
acls:
manageSystemACLs: false
bootstrapToken:
secretName: bootstrap-token
secretKey: token
client:
enabled: true
exposeGossipPorts: true
join:
- 'xxx.xx.xx.xx'
- 'xxx.xx.xx.xx'
- 'xxx.xx.xx.xx'
externalServers:
enabled: true
hosts: ["xxx.xx.xx.xx", "xxx.xx.xx.xx", "xxx.xx.xx.xx"]
Checking the logs of one of the clients, I am getting a lot of error=“rpc error making call: rpc error making call: Permission denied”
==> Starting Consul agent...
Version: '1.9.4'
Node ID: 'bxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
Node name: 'node1'
Datacenter: 'dc1' (Segment: '')
Server: false (Bootstrap: false)
Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: 8502, DNS: 8600)
Cluster Addr: xxx.xx.xx.xx (LAN: 8301, WAN: 8302)
Encrypt: Gossip: true, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false
==> Log data will now stream in as it occurs:
2021-05-19T09:47:01.605Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: node1 xxx.xx.xx.xx
2021-05-19T09:47:01.605Z [INFO] agent.router: Initializing LAN area manager
2021-05-19T09:47:01.605Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=udp
2021-05-19T09:47:01.605Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=tcp
2021-05-19T09:47:01.605Z [INFO] agent: Starting server: address=[::]:8500 network=tcp protocol=http
2021-05-19T09:47:01.606Z [WARN] agent: DEPRECATED Backwards compatibility with pre-1.9 metrics enabled. These metrics will be removed in a future version of Consul. Set `telemetry { disable_compat_1.9 = true }` to disable them.
2021-05-19T09:47:01.606Z [INFO] agent: Started gRPC server: address=[::]:8502 network=tcp
2021-05-19T09:47:01.606Z [INFO] agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean gce k8s linode mdns os packet scaleway softlayer tencentcloud triton vsphere"
2021-05-19T09:47:01.606Z [INFO] agent: Joining cluster...: cluster=LAN
2021-05-19T09:47:01.606Z [INFO] agent: (LAN) joining: lan_addresses=[xxx.xx.xx.xx, xxx.xx.xx.xx, xxx.xx.xx.xx]
2021-05-19T09:47:01.607Z [INFO] agent: started state syncer
==> Consul agent running!
2021-05-19T09:47:01.607Z [WARN] agent.router.manager: No servers available
2021-05-19T09:47:01.607Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No known Consul servers"
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: enode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: enode3 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [WARN] agent.client.memberlist.lan: memberlist: Refuting a dead message (from: node1)
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: enode2 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: inode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: nnode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: inode3 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client: adding server: server="inode1 (Addr: tcp/xxx.xx.xx.xx:8300) (DC: dc1)"
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: node2 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client: adding server: server="inode3 (Addr: tcp/xxx.xx.xx.xx:8300) (DC: dc1)"
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: dnode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: inode2 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client: adding server: server="inode2 (Addr: tcp/xxx.xx.xx.xx:8300) (DC: dc1)"
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: mnode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: knode1 xxx.xx.xx.xx
2021-05-19T09:47:01.610Z [INFO] agent.client.serf.lan: serf: EventMemberJoin: anode1 xxx.xx.xx.xx
2021-05-19T09:47:01.615Z [INFO] agent: (LAN) joined: number_of_nodes=3
2021-05-19T09:47:01.615Z [INFO] agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=3
2021-05-19T09:47:04.331Z [ERROR] agent.client: RPC failed to server: method=Catalog.Register server=xxx.xx.xx.xx:8300 error="rpc error making call: Permission denied"
2021-05-19T09:47:04.332Z [WARN] agent: Node info update blocked by ACLs: node=bxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx accessorID=
2021-05-19T09:47:06.466Z [ERROR] agent.client: RPC failed to server: method=Catalog.Register server=xxx.xx.xx.xx:8300 error="rpc error making call: rpc error making call: Permission denied"
2021-05-19T09:47:06.466Z [WARN] agent: Node info update blocked by ACLs: node=bxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx accessorID=
2021-05-19T09:47:19.056Z [ERROR] agent.client: RPC failed to server: method=Coordinate.Update server=xxx.xx.xx.xx:8300 error="rpc error making call: rpc error making call: Permission denied"
2021-05-19T09:47:19.057Z [WARN] agent: Coordinate update blocked by ACLs: accessorID=
I am using the main bootstrapToken(global management).
Checking the docs for consul-K8s with external servers (Consul Servers Outside of Kubernetes - Kubernetes | Consul by HashiCorp). The method shown here is with manageSystemACLs: true, is it possible to run it without it? creating the needed tokens manually.
consul agent: 1.9.4
consul-k8s: 0.31.1
Thank you 