Hello all,
I am running a consul server outside of K8s. I have separatet k8s cluster and all nodes are reachable. I am not running any ACL or TLS at the moment. I am getting the below error and the injector pod is failing in K8s.
ERROR:
2024-08-31T12:33:30.189Z [INFO] consul-server-connection-manager.consul-server-connection-manager: trying to connect to a Consul server
2024-08-31T12:33:30.296Z [ERROR] consul-server-connection-manager.consul-server-connection-manager: connection error: error="failed to discover Consul server addresses: failed to resolve DNS name: consul-consul-server.consul.svc: lookup consul-consul-server.consul.svc on 10.96.0.10:53: no such host"
It seems even if I give the externalServer host IP it doesn’t work. Am I missing something here?
My helm values for k8s
global:
enabled: false
tls:
enabled: false
externalServers:
enabled: true
hosts: ["192.168.60.10"]
server:
enabled: false
I installed consul using helm
helm install consul hashicorp/consul --namespace consul -f helm-values.yaml
The resources in K8s
NAME READY STATUS RESTARTS AGE
pod/consul-consul-connect-injector-bf57cf9b4-tzxcg 0/1 Running 0 30s
pod/consul-consul-gateway-resources-q44f7 0/1 Completed 0 2m42s
pod/consul-consul-webhook-cert-manager-7c656f9967-hsr8v 1/1 Running 0 30s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/consul-consul-connect-injector ClusterIP <none> 443/TCP 30s
service/consul-consul-dns ClusterIP <none> 53/TCP,53/UDP 30s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/consul-consul-connect-injector 0/1 1 0 30s
deployment.apps/consul-consul-webhook-cert-manager 1/1 1 1 30s
NAME DESIRED CURRENT READY AGE
replicaset.apps/consul-consul-connect-injector-bf57cf9b4 1 1 0 30s
replicaset.apps/consul-consul-webhook-cert-manager-7c656f9967 1 1 1 30s10.103.254.16610.97.215.246
When I check the logs in the inject pod it says below
k logs -n consul pod/consul-consul-connect-injector-bf57cf9b4-tzxcg
2024-08-31T12:33:30.189Z [INFO] consul-server-connection-manager.consul-server-connection-manager: trying to connect to a Consul server
2024-08-31T12:33:30.296Z [ERROR] consul-server-connection-manager.consul-server-connection-manager: connection error: error="failed to discover Consul server addresses: failed to resolve DNS name: consul-consul-server.consul.svc: lookup consul-consul-server.consul.svc on 10.96.0.10:53: no such host"
I can ping to the consul server VM IP from K8s pod, also I could access services
curl http://192.168.60.10:8500/v1/catalog/services
{"consul":[]}
I sincerely appreciate if some one could kindly tell me what is wrong with this setup.
Thank you!
PS: I also checked the deployment consul-consul-connect-injector and it has below ENV variables
Environment:
NAMESPACE: (v1:metadata.namespace)
POD_NAME: (v1:metadata.name)
CONSUL_ADDRESSES: consul-consul-server.consul.svc
CONSUL_GRPC_PORT: 8502
CONSUL_HTTP_PORT: 8500
CONSUL_DATACENTER: dc1
CONSUL_API_TIMEOUT: 5s