Vault Cluster with ELB - Failing health checks, not relaying to nodes

I am using this repo to set up a Vault/Consul cluster. I have it deployed in AWS and I’m using the vault-elb module to set up the Elastic Load Balancer. This all spins up without a problem.

Now that the cluster is active, I’m having trouble figuring out who isn’t doing their job. The load balancer is up, but if I ping or browse to the DNS or IP of the ELB, it just spins. The ELB shows that all the nodes are failing their health checks, however if I browse to one of the Vault servers directly and tack on the health check string, I at least get a response:


I think I’m confused on how to customize the domain name being used if you don’t want to point your users to vault.service.consul (or am I confused on that point? Is that just for the backend, and the ELB config is separate?). Our domain is (EDIT)NOT managed in AWS, so I had our sysadmin point a CNAME record for to the ELB DNS name. As such, I didn’t enable the create_dns_entry option to do anything in Route53. What is the proper configuration to use here?

Would help if you post your Vault configs, and also the log should be guiding to finding the root cause of the problem.

I think I see the issue here - if I set the create_dns_entry option to False, none of the subsequent variables that configure the domain name are used. So my question is a little more distilled into: How do you configure vault-elb module for your domain if you’re not using Route53?

A quick update:
I’ve been working on this all evening, and I thought I found an issue because I didn’t use the private-tls-cert module correctly when setting up my AMI, so I was getting X509 errors if I tried to look at vault status and that appears to be where you would set up at least one reference to the desired domain you want to use.

Once I fixed that, I’m still seeing the same thing. I can connect to the Vault servers directly via IP:8200 but the ELB is still showing all the Vault nodes are OutOfService.

This is a shot in the dark, but do I need to unseal the vault for some reason in order for the ELB to talk to the consul service properly?

Bump, any updates here?
We are facing the same issue with the ELB health checks.

@devops-jared best if you post a new thread with your configs/specific issues/logs.