Consul web UI TLS enabled

tmiroslav · May 26, 2021, 12:38pm

We are using Consul which is SSL enabled when it comes to consul agent-server communication ( built-in CA utilized). But when is comes to Consul Web UI, there is no SSL there. it’s pure HTTP. Is there possibility to enable SSL for UI through consul config, by providing certs which would be used for auth from browser side. Something similar to Vault SSL setup for example?
Is only option to put something in front for auth, like some proxy which would play a role in Web auth?

Thank you!

Ranjandas · May 27, 2021, 2:42am

Hi @tmiroslav,

Consul will not enable TLS for the HTTP API/UI unless you assign a port for https. You can do this by configuring the ports option.

ports {
    "http" = -1 #if you want to disable HTTP 
    "https" = 8501 # recommended for https by convention as some tooling will work automatically with this.
}

This learn guide explains on how to secure your cluster with TLS encryption: Secure Consul Agent Communication with Encryption and Certificates | Consul - HashiCorp Learn

Make sure that you also read this section Secure Consul Agent Communication with Encryption and Certificates | Consul - HashiCorp Learn if you don’t want your UI to have Client Cert authentication.

tmiroslav · May 27, 2021, 11:41am

@Ranjandas

Thank you for this very useful inputs.

I suppose same could be configured by using Vault CA, for server and client certs generation? And later on just use that CA in the browser to access Consul server UI.

Ranjandas · May 27, 2021, 11:56am

Yes, you are right. The following documentation should help you with it: Generate mTLS Certificates for Consul with Vault | Consul - HashiCorp Learn

george.bell · May 27, 2021, 9:27pm

If you disable http, does that mean you will have to create a certificate signed by the CA that signed the consul server and client certs.

george.bell · May 27, 2021, 9:42pm

Sorry, I should have included the config.json I am using

{
“advertise_addr”: “20.0.3.33”,
“bind_addr”: “20.0.3.33”,
“bootstrap_expect”: 3,
“client_addr”: “0.0.0.0”,
“datacenter”: “tmpcenter”,
“data_dir”: “/data/1/consul”,
“domain”: “consul”,
“enable_local_script_checks”: true,
“enable_script_checks”: false,
“disable_remote_exec”: true,
“dns_config”: {
“enable_truncate”: true,
“only_passing”: true
},
“enable_syslog”: true,
“encrypt”: “###############################”
“leave_on_terminate”: true,
“log_file” : “/data/1/log/”,
“log_level”: “DEBUG”,
“node_name”: “consul-33”,
“rejoin_after_leave”: true,
“retry_join”: [
“IP13”,“IP238”,“IP.33”
],
“server”: true,
“ui_config”: {
“enabled”: true
},
“verify_incoming”: false,
“verify_incoming_rpc”: true,
“verify_outgoing”: true,
“verify_server_hostname”: true,
“ca_file”: “/data/1/certs/CACERT.pem”,
“cert_file”: “/data/1/certs/consul-33.cert”,
“key_file”: “/data/1/certs/consul-33key”,
“ports”: {
“http”: -1,
“https”: 8501
}
}

Note: the openssl tls docs said you could set verify incoming to false

Ranjandas · May 28, 2021, 1:08am

Hi @george.bell

Welcome to the Forum!

There are two configs that you should be aware of here:

verify_incoming: true - This is applicable to both RPC and HTTP. If you set this to true, clients talking to the Consul server (via RPC and HTTP) will have to authenticate using client certificates.
verify_incoming: false + verify_incoming_rpc: true - The combination of these configs gives you an option to have mTLS authentication only for RPC connections. This means you can connect to the Consul UI/API without having a Client certificate to prove the client’s identity but RPCs would still need it.

Hope this answers your question. Your config seems to be using it right.

george.bell · May 28, 2021, 11:19am

Thanks @Ranjandas most appreciated.

After rereading the documentation, I think my first problem is that when implementing tls I need to only configure 1 or 2 agents with ui_config {enabled=true}. So I have set all agents back to the recommended verify_incoming:true, removed verify_incoming_rpc entry from config and and set the ui_config enabled to false on all agents in the data center.

I am assuming, now I need to deploy a ui agent, register the service as ui and then change my proxy to access the ui agent with the verify_incoming:false and verify_incoming_rpc: true.

I had this working (tls disabled) in consul 1.1 in a dev environment, and the proxy contained

I am thinking those are probably pretty out of date, is there a place in the documentation where I can find the recent url as static appears to have been permanently moved. Or, more likely, I am still doing something wrong

tmiroslav · May 28, 2021, 11:22am

I am also interested if same cert used in Consul client side would be fine to use in Web browser to access Consul UI? In our case, only Ca cert is used in client side, and I am assuming that it could be fine to import that one in Web browser.

george.bell · May 28, 2021, 11:31am

@tmiroslav I use aws certificate manager to create the private ca and then had all agents in cluster create a cert upon instance creation. I followed the same process for a user cert if you will that I then converted to p12 and stuck in my browser. That appeared to work just fine. Not sure if that truly answers your question, but I think you can use the same cert.

Ranjandas · May 28, 2021, 11:53am

If you are only configuring ca_cert on the client-side, then you are probably using the AutoEncrypt feature, where the Certificate and Private key is generated and distributed by the Consul server to the clients.

If you are setting verify_incoming: true, then you should generate a private/public key pair signed by a CA your Consul server trusts. Also, convert it to a format that your browser can import. Just having a CA certificate in the browser isn’t enough for doing Client Cert authentication.

Ref: Secure Consul Agent Communication with Encryption and Certificates | Consul - HashiCorp Learn

tmiroslav · May 30, 2021, 3:22pm

@Ranjandas
Thank you! You are right here, we use AutoEncrypt setup with verify_incoming: true. In that case, we would need to create new crt/key client pair and to chain it somehow together with ca cert file, and import it then in browser. We will need to test if it work this way. It’s a bit different setup with Vault certs, but there we used just Vault root CA to import in browser, and we could open Vault GUI.

Ranjandas · May 30, 2021, 3:27pm

You can achieve the same as what you did with Vault UI for Consul UI by setting verify_incoming: false and verify_incoming_rpc: true. With this, you only need the CA certificate in your browser for the browser to trust Consul UI.

tmiroslav · May 30, 2021, 5:32pm

@Ranjandas
Ok, I understand.
In our scenario, we are having 3 consul servers running in cluster, behind load balancer. I assume we will have to enable UI settings in all 3 cluster nodes, as LB is probably doing some round robin.

tmiroslav · June 1, 2021, 9:10pm

@Ranjandas
I have managed to open Consul UI by setting parameters verify_incoming: false and verify_incoming_rpc: true and importing CA certificate in browser like you proposed(even though certificate was created for domain which is different from the one I used to access Consul UI). But now , as I disabled http in Consul server config, I faced issue with executing local commands in consul terminal (like consul members). I got
Get "https://127.0.0.1:8500/v1/agent/members?segment=_all": dial tcp 127.0.0.1:8500: connect: connection refused.
And when I try to do export like
export CONSUL_HTTP_ADDR=https://consul.service.consul:8501
it is complaining that certificate is not valid for domain consul.service.consul
Do I have to issue new certificates? And if I have, which domain I should issue it for now, knowing that HTTPs is enabled and HTTP is disabled in my config?
Thank you!

george.bell · June 1, 2021, 9:38pm

I had to set the following env for cli to work

CONSUL_HTTP_ADDR
CONSUL_CACERT=consul ca cert file
CONSUL_CLIENT_CERT=client cert file signed by consul ca cert
CONSUL_CLIENT_KEY=client cert key

george.bell · June 1, 2021, 9:43pm

I have added the following to the agent that is running the ui, but the content appears to still be served from /ui/

“ui_config”: {
“enabled”:“true”,
“content_path”:"/consului/"
}

I am trying to use httpd 2.4 and there is a conflict when vault and consul are included in httpd.conf in the /ui/ dir. Vault does not appear to have a fix, so was trying for change /ui/assets for consul to /consului/assets

Can this be done?

tmiroslav · June 2, 2021, 8:52am

@george.bell
Thank you for sharing that.
My problem is actually fact that, in my existing setup, where consul auto-encrypt functionality is used, certs are created using SAN which does not point to actual Consul DNS name (SAN used when certs are created is *.frame.consul, and Consul service is accessible on consul.service.consul from internal and consul.xxx.yyy.com from external). So, whatever CONSUL_HTTP_ADDR is exported it, either does not comply with certs SAN name (when
export CONSUL_HTTP_ADDR=https://consul.service.consul:8501),
or it can not find Consul service ( when
export CONSUL_HTTP_ADDR=https://consul.frame.consul:8501 , as obviously Consul is not running there)
It seems to me that certs recreation and its replacement in existing Consul setup will have to be done.

george.bell · June 2, 2021, 10:51am

Ignore this post, it worked after I cleared the httpd cache.

Ranjandas · June 2, 2021, 10:59am

@tmiroslav, on Consul client agents, if you are using auto-encrypt, the certificates are issued by Consul Connect CA. If you are talking to Consul agent from within the server try setting CONSUL_HTTP_ADDR=https://127.0.0.1:8501. The certs usually will have 127.0.0.1 in the SAN.

If you are interacting with the Consul API from outside the server, make sure you use the right hostname, and your cert should include the hostname in the SAN’s for the client to trust the server.

Could you please let me know if your certs are created using consul tls utility?

Topic		Replies	Views
TLS verification and web UI Consul	4	1195	September 24, 2019
// , Consul Server does Not Prompt Web Clients for Client TLS Certificates Consul	0	263	November 6, 2019
Why do consul TLS guides leave client in state where verify_incoming = false? Consul	3	355	October 19, 2022
Vault config with tls-enabled Consul as a storage backend Consul	0	320	July 17, 2020
Enabling tls - certificate signed by unknown authority Consul	6	1916	February 11, 2022

Consul web UI TLS enabled

Related topics