Consul servers outside kubernetes

Consul
,
1

Hi ,

I have consul servers which are running on local VM’s:

[root@vm1 autoinstall]# consul members
Node          Address             Status  Type    Build  Protocol  DC   Segment
vm1  10.225.20.101:8301  alive   server  1.8.3  2         dc1  <all>
vm2  10.225.20.102:8301  alive   server  1.8.3  2         dc1  <all>
vm3  10.225.20.103:8301  alive   server  1.8.3  2         dc1  <all>

I am trying to join agents from my kubernetes cluster:

[root@k8masterg2m1 autoinstall]# kubectl get po
NAME                  READY   STATUS    RESTARTS   AGE
consul-consul-4k6hd   0/1     Running   0          25m
consul-consul-89z2k   0/1     Running   0          25m
consul-consul-f6nd4   0/1     Running   0          25m
consul-consul-k4tj5   0/1     Running   0          25m
consul-consul-krj8k   0/1     Running   0          25m

which is throwing error like below

[root@k8masterg2m1 autoinstall]# kubectl logs -f consul-consul-89z2k
==> Starting Consul agent...
           Version: '1.8.2'
           Node ID: '1a5d33e2-84bb-1708-09a9-966ade1a6a9b'
         Node name: 'k8workerg2w3'
        Datacenter: 'dc1' (Segment: '')
            Server: false (Bootstrap: false)
       Client Addr: [0.0.0.0] (HTTP: -1, HTTPS: 8501, gRPC: 8502, DNS: 8600)
      Cluster Addr: 10.244.118.199 (LAN: 8301, WAN: 8302)
           Encrypt: Gossip: false, TLS-Outgoing: true, TLS-Incoming: false, Auto-Encrypt-TLS: true
==> Log data will now stream in as it occurs:
    2020-08-28T17:20:12.620Z [INFO]  agent.client.serf.lan: serf: EventMemberJoin: k8workerg2w3 10.244.118.199
    2020-08-28T17:20:12.718Z [WARN]  agent.client.manager: No servers available
    2020-08-28T17:20:13.019Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-0.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:13.057Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-1.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:13.061Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-2.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:13.061Z [WARN]  agent.client: retrying AutoEncrypt: retry_interval=40.239313296s
    2020-08-28T17:20:53.333Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-0.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:53.368Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-1.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:53.406Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-2.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:20:53.406Z [WARN]  agent.client: retrying AutoEncrypt: retry_interval=36.411590523s
    2020-08-28T17:21:29.856Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-0.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:21:29.896Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-1.consul-consul-server.default.svc on 10.96.0.10:53: no such host"
    2020-08-28T17:21:29.935Z [WARN]  agent.client: AutoEncrypt resolveAddr failed: error="lookup consul-consul-server-2.consul-consul-server.default.svc on 10.96.0.10:53: no such host"

can anybody please give a suggestion, how can I solve this.

I have services in Kubernetes as shown below:

[root@k8masterg2m1 autoinstall]# kubectl get svc
NAME                TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)         AGE
consul-consul-dns   ClusterIP   10.99.78.91   <none>        53/TCP,53/UDP   34m
kubernetes          ClusterIP   10.96.0.1     <none>        443/TCP         86m

This is the configuration enabled for helm chart:

[root@k8masterg2m1 autoinstall]# cat config.yaml
global:
  tls:
    enabled: true
    enableAutoEncrypt: true
externalServers:
  enabled: true
  hosts: ["<vm1_ip>:8301","<vm2_ip>:8301","<vm3_ip>:8301"]
dns:
  enabled: true

Any suggestion is appreciated.