Consul web UI TLS enabled

Consul
21

I went the route of uses AWS certificate manager to create the certs instead of auto encryption so not sure there. I leveraged the user data portion of Cloudformation to install consul and would create certs for all instances using consul with SAN. This is more for a management network, so I would have services like, consul, vault, jenkins, grafana and elk. I would run the:

aws acm request-certificate --domain-name $thisModule.node.consul --validation-method DNS --subject-alternative-names $thisModule.node.$DATACENTER.consul $thisModule.service.consul --certificate-authority-arn subordinateCAarn --tags Key=Name,Value=$thisModule --output text

That would give me back the arn of the cert created and then I would export the cert using the “aws acm export-certificat” command, and make a keystore, truststore or whatever the service required from that.

I elected this route so that if let’s say my jenkins worker died, I have it on an ASG and upon instance creation it would firs look to see if the jenkins cert existed, if not, it would create it, but it always have to export to create cert/jks/ etc.

Hashicorp has good documentation on this, in the Openssl TLS section. Sorry I am not providing a link.

Last thing. I have a three node consul cluster and I was having issues wtih consulUI. SO I launched an ec2-instance that served as the consul ui. Meaning, all other services including the consul servers where configured with
“ui_config”: {
“enabled”:“false”
}
“verify_incoming”: true
“verify_outgoing”: true

Then on the server that I called ConsulUI I had these settings

“ui_config”: {
“enabled”:“true”,
“content_path”:"/consului/"
},
“verify_incoming”: false,
“verify_incoming_rpc”: true,
“verify_outgoing”: true

I had to add the content_path setting above because I access all services behind a reverse proxy but consul and vault both use /ui/ for content path and that was causing issues. Consul supports the content_path but Vault does not, well at least 1.4 does not. I am using 1.9.5 consul and will upgrade vault when customer gives approval.

Not sure if this helps, but putting it out there as it did take a while reading consul docs to find all the right combinations

22

@tmiroslav I did forget one last thing in that post. If you need the IPs in the cert, you are going to have to go the openssl route as amazon does not appear to support IP in san. I could be wrong, but that appears to be the case.

23

@Ranjandas Yes, my certs are created using consul tls. And more specifically, certs are created with these commands:
consul tls ca create -name-constraint=true -additional-name-constraint=consul-xxx.yyyy.zzzz.mmm.com -domain=consul
consul tls cert create -server -dc=frame -additional-dnsname=consul-xxx.yyyy.zzzz.mmm.com
And yes, I am connected to server node. When try with CONSUL_HTTP_ADDR=https://127.0.0.1:8501
This is the error I am facing:

Error retrieving members: Get “https://127.0.0.1:8501/v1/agent/members?segment=_all”: x509: cannot validate certificate for 127.0.0.1 because it doesn’t contain any IP SANs

If I set like CONSUL_HTTP_ADDR=https://localhost:8501, the error is following:

Error retrieving members: Get “https://localhost:8501/v1/agent/members?segment=_all”: x509: certificate is valid for *.frame.consul, not localhost

It looks like , way CA was created does not giving me an options to do anything, but to create new certs from the begging and replace existing one. What do you think?

24

I executed the same command as yours and inspected the certs with OpenSSL. I can see both 127.0.0.1 and localhost in the list of SANs.

Could you please verify your certs. Are the certs currently active taken from the result of the above commands?

Even the consul tls cli help says both 127.0.0.1 and localhost will be always included even when you use additional name constraints.

  -additional-dnsname=<value>
     Provide an additional dnsname for Subject Alternative Names.
     localhost is always included. This flag may be provided multiple
     times.

  -additional-ipaddress=<value>
     Provide an additional ipaddress for Subject Alternative Names.
     127.0.0.1 is always included. This flag may be provided multiple
     times.
25

@Ranjandas Yes, you were right. There was some issue with my certs. After I recreated it using consul tls it worked as expected. Thank you a lot for assisting here!

1 Like
26

@Ranjandas
After setting up consul per instructions above, web UI is working as expected.
But, when it comes to consul CLI for https, it’s strange that export CONSUL_HTTP_ADDRESS variable is not working as expected. Even if I export https port 8501, it is trying to access http port 8500 and being refused - what is expected.

export CONSUL_HTTP_ADDRESS=https://127.0.0.1:8501
[centos@frame-consul10-242-56-133 ~]$ consul members
Error retrieving members: Get "http://127.0.0.1:8500/v1/agent/members?segment=_all": dial tcp 127.0.0.1:8500: connect: connection refused

Also if I do this different way, it works fine:

[centos@frame-consul10-242-56-133 ~]$ consul members -http-addr=https://127.0.0.1:8501
Node                                                      Address             Status  Type    Build  Protocol  DC     Segment
frame-consul10-242-56-133                                 10.242.56.133:8301  alive   server  1.8.3  2         frame  <all>
frame-consul10-242-57-114                                 10.242.57.114:8301  alive   server  1.8.3  2         frame  <all>
frame-consul10-242-58-245                                 10.242.58.245:8301  alive   server  1.8.3  2         frame  <all>

Seems like there is problem with export command.

27

@tmiroslav, you have got the environment variable wrong. It’s CONSUL_HTTP_ADDR.

Ref: Commands | Consul by HashiCorp

28

Ah… That’s correct!
Thanx a lot!