Consul service DNS resolution not working

I have a few new clusters running Consul v1.9.1

Consul is definitely up, the UI works. I can register services, but using the DNS interface for discovery just yields the below.

$ dig @127.0.0.1 -p 8600 consul.service.consul ANY

; <<>> DiG 9.16.1-Ubuntu <<>> @127.0.0.1 -p 8600 consul.service.consul ANY
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 55212
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;consul.service.consul. IN ANY

;; AUTHORITY SECTION:
consul. 30 IN SOA ns.consul. hostmaster.consul. 1614886998 3600 600 86400 30

;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Thu Mar 04 19:43:18 UTC 2021
;; MSG SIZE rcvd: 100

OS : Ubuntu 20.04

Happy to provide any information required. This is absolutely baffling as I can see from the logs :

[DEBUG] agent.dns: request served from client: name=consul.service.sandbox.consul. type=A class=IN latency=1.653581ms client=127.0.0.1:52840 client_network=udp

even stranger is that recursion works fine

; <<>> DiG 9.16.1-Ubuntu <<>> @localhost -p 8600 www.google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48599
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.google.com. IN A
;; ANSWER SECTION:
www.google.com. 253 IN A 74.125.193.103
www.google.com. 253 IN A 74.125.193.105
www.google.com. 253 IN A 74.125.193.104
www.google.com. 253 IN A 74.125.193.147
www.google.com. 253 IN A 74.125.193.99
www.google.com. 253 IN A 74.125.193.106
;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Thu Mar 04 17:05:44 UTC 2021
;; MSG SIZE rcvd: 128

Hi @Nexus357ZA,

Do you have ACLs enabled on this cluster? If yes, refer to this guide Secure Consul with Access Control Lists (ACLs) | Consul - HashiCorp Learn

You will have to appropriate permissions to the anonymous token to get the DNS interface working.

2 Likes

This was exactly the issue.