Consul retry_join is setting self host ip and fail?

Consul
1

Hi I’m setting up a cluster on DigitalOcean and came up with an interesting situation.

if there’s only 1 consul server and retry_join is set. using Cloud Auto-join

{
  "retry_join": ["provider=digitalocean region=... tag_name=... api_token=..."]
}

it was returning the VM self IP as it is the only one with the existing tags, name, etc…
this was resulting in an error of:

 error=
  | 1 error occurred:
  | \t* Failed to join 10.116.0.2:8301: dial tcp 10.116.0.2:8301: connect: connection refused
  |

When I removed retry_join parameter it all worked just fine. is this a bug?

CONSUL VERSION:

Consul v1.17.0
Revision 4e3f428b
Build Date 2023-11-03T14:56:56Z
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

2

you only have one single consul server in your cluster, so I believe this is expected because the server is already running and cannot form a cluster with itself.

1 Like
3

Hi @andresogando10,

You are having this issue most probably because your Consul agent is not listening on the IP 10.116.0.2.

Try setting your --bind-addr to the primary interface of your VM (instead of 127.0.0.1) and restart the agent and this should make it work.

ref: Agents - Configuration File Reference | Consul | HashiCorp Developer

You can see this behaviour here; in this example, I am binding to 127.0.0.1 but trying to join to the host IP.

ref: Single Node Cluster - Serf Connection Refused - asciinema

And here when I bind to the host IP, it works.

ref: untitled - asciinema

I hope this helps.

1 Like
4

I had the advertise_addr wrong and that’s why it was failing.

this is my consul config file that is working just fine now.

log_level  = "DEBUG"
datacenter = "dc1"
data_dir   = "/opt/consul"
log_file  = "/var/log/consul.log"
# encrypt    = ""
server = true
# bootstrap_expect = $BOOTSTRAP_EXPECT
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
advertise_addr = "$LOCAL_IP"
leave_on_terminate = true

retry_join = ["provider=digitalocean tag_name=...  tag_value=...  api_token=<TOKEN> "]

acl {
  enabled                  = true
  default_policy           = "deny"
  enable_token_persistence = true
  enable_token_replication = true
  down_policy = "extend-cache"
}

connect {
  enabled = true
}

addresses {
  grpc = "0.0.0.0"
  https = "0.0.0.0"
  dns = "0.0.0.0"
}

# ports {
#   grpc_tls  = 8503
#   grpc = 8502
#   http  = 8500
#   serf_lan = 8301
#   serf_wan = 8302
#   https = ${HTTPS_PORT}
#   dns   = ${DNS_PORT}
# }

# DNS recursors
# recursors = ["${DNS_RECURSOR}"]

ui_config {
  enabled = true
}

#  auto_encrypt {
#    allow_tls = true
#  }

#  tls {
#    defaults {
#      ca_file   = "$CONSUL_CACERT"
#      cert_file = "$CONSUL_CLIENT_CERT"
#      key_file  = "$CONSUL_CLIENT_KEY"

#      verify_incoming = true
#      verify_outgoing = true
#    }
#    internal_rpc {
#      verify_server_hostname = true
#    }
#  }
5

Yep I had to set up the advertise_addr and it started working. but this docs are very useful to someone else who is stuck in something similar. thanks!