Vault Cluster with ELB - Failing health checks, not relaying to nodes

Vault
1

Hello,
I am using this repo to set up a Vault/Consul cluster. I have it deployed in AWS and I’m using the vault-elb module to set up the Elastic Load Balancer. This all spins up without a problem.

Now that the cluster is active, I’m having trouble figuring out who isn’t doing their job. The load balancer is up, but if I ping or browse to the DNS or IP of the ELB, it just spins. The ELB shows that all the nodes are failing their health checks, however if I browse to one of the Vault servers directly and tack on the health check string, I at least get a response:

{"initialized":false,"sealed":true,"standby":true,"performance_standby":false,"replication_performance_mode":"unknown","replication_dr_mode":"unknown","server_time_utc":1606856273,"version":"1.6.0"}

I think I’m confused on how to customize the domain name being used if you don’t want to point your users to vault.service.consul (or am I confused on that point? Is that just for the backend, and the ELB config is separate?). Our domain is (EDIT)NOT managed in AWS, so I had our sysadmin point a CNAME record for vault.company.com to the ELB DNS name. As such, I didn’t enable the create_dns_entry option to do anything in Route53. What is the proper configuration to use here?

2

Would help if you post your Vault configs, and also the log should be guiding to finding the root cause of the problem.

3

I think I see the issue here - if I set the create_dns_entry option to False, none of the subsequent variables that configure the domain name are used. So my question is a little more distilled into: How do you configure vault-elb module for your domain if you’re not using Route53?

4

A quick update:
I’ve been working on this all evening, and I thought I found an issue because I didn’t use the private-tls-cert module correctly when setting up my AMI, so I was getting X509 errors if I tried to look at vault status and that appears to be where you would set up at least one reference to the desired domain you want to use.

Once I fixed that, I’m still seeing the same thing. I can connect to the Vault servers directly via IP:8200 but the ELB is still showing all the Vault nodes are OutOfService.

This is a shot in the dark, but do I need to unseal the vault for some reason in order for the ELB to talk to the consul service properly?

5

Bump, any updates here?
We are facing the same issue with the ELB health checks.

6

@devops-jared best if you post a new thread with your configs/specific issues/logs.