Hello,
I’ve some troubles connecting Consul from Vault and cannot be sure which certificates must be used to connect Consul.
consul-server-cert
consul-ca-key
consul-ca-cert
When I check root certificate in Consul, I am not able to see the same cert in any Kubernetes secret.
kubectl exec consul-consul-server-0 -n consul -- curl -sk https://localhost:8501/v1/connect/ca/roots | jq -r .Roots[0].RootCert)
I am so confused, please advise.
Thank & Regards
aram
December 22, 2021, 11:46am
2
You’re confusing multiple different things. CA is the list of CAs that are trusted within Consul for outbound connections.
When vault tries to setup a TLS connection to Consul, then all it needs to know is that there is a valid certificate on the other end. As long as the CA that is used to generate the certificate in the “listen” block is a trusted certificate, and that the certificate includes the proper hostname, fqdn, SANs of the consul server that you defined in your vault storage
block then it’ll work.
@aram thanks for your reply
how can I check or how can I be sure Vault can or cannot connect to Consul?
aram
December 23, 2021, 12:48am
4
Start vault, if it can’t connect you won’t be able to start it up or initialize it.
vault status
shows you what the configuration of the backend storage is.
@aram I had some progress…
yesterday I was able to enable tls_skip_verify and Vault connected to Consul.
it’s ok.
Next step, I removed tls_skip_verify and now I’m at the same point. Vault cannot connect to tls enabled Consul.
NAME TYPE DATA AGE
consul-consul-acl-replication-acl-token Opaque 1 9h
consul-consul-bootstrap-acl-token Opaque 1 9h
consul-consul-ca-cert Opaque 1 9h
consul-consul-ca-key Opaque 1 9h
consul-consul-client-acl-token Opaque 1 9h
consul-consul-client-token-4tbv6 kubernetes.io/service-account-token 3 9h
consul-consul-gossip-encryption-autogenerate-token-pwhtg kubernetes.io/service-account-token 3 9h
consul-consul-gossip-encryption-key Opaque 1 9h
consul-consul-server-acl-init-cleanup-token-zpgtw kubernetes.io/service-account-token 3 9h
consul-consul-server-acl-init-token-9wk6s kubernetes.io/service-account-token 3 9h
consul-consul-server-cert kubernetes.io/tls 2 9h
consul-consul-server-token-42szp kubernetes.io/service-account-token 3 9h
consul-consul-tls-init-token-49wt4 kubernetes.io/service-account-token 3 9h
default-token-52xgx kubernetes.io/service-account-token 3 9h
sh.helm.release.v1.consul.v1 helm.sh/release.v1 1 9h
The secrets of Consul are above.
I use following secrets with/without consul-consul-server-cert, Vault cannot connect to tls enabled Consule securely.
tls_ca_file = "/vault/userconfig/consul-consul-ca-cert/tls.crt"
tls_key_file = "/vault/userconfig/consul-consul-ca-key/tls.key"
tls_cert_file = "/vault/userconfig/consul-consul-server-cert/tls.crt"
The question is that which certificates must be used to tls enabled Consul by Vault? Which certificates are the correct certificates to connect tls enabled Consul?
Thanks & Best Regards
Hello @ishustava @ishustava1
Reagarding the discussion here , which certificate is the correct certificate to connect Consul from Vault?
That one;
kubectl create secret generic consul-client-ca --from-literal=ca="$(kubectl exec consul-server-0 -- curl -sk https://localhost:8501/v1/connect/ca/roots | jq -r .Roots[0].RootCert)"
or that one?
consul-consul-ca-cert/tls.crt
Also please check following dialogs:
Great, thanks @lkysow . I have proven the below workflow. Perhaps it might help in official documenting of this use case or, at the very least, help others who might stumble upon this post!
Create a Kubernetes secret named consul with a key named CONSUL_GOSSIP_ENCRYPTION_KEY and an appropriate encryption key value.
Generate value using consul keygen
Install the hashicorp/consul Helm chart with an values-override.yaml, such as below:
global:
datacenter: sandbox
gossipEncryption:
se…
I am having the same issue. How were you able to solve your issue?
opened 07:37AM - 22 Dec 21 UTC
closed 03:54PM - 17 Aug 22 UTC
bug
chart
vault-server
Hello,
These are all secrets in Consul namespace:
```
NAME … TYPE DATA AGE
consul-consul-acl-replication-acl-token Opaque 1 15h
consul-consul-bootstrap-acl-token Opaque 1 15h
consul-consul-ca-cert Opaque 1 15h
consul-consul-ca-key Opaque 1 15h
consul-consul-client-acl-token Opaque 1 15h
consul-consul-client-token-zrwmx kubernetes.io/service-account-token 3 15h
consul-consul-gossip-encryption-autogenerate-token-s4sj5 kubernetes.io/service-account-token 3 11h
consul-consul-gossip-encryption-key Opaque 1 15h
consul-consul-server-acl-init-cleanup-token-7qtm4 kubernetes.io/service-account-token 3 15h
consul-consul-server-acl-init-token-hpttl kubernetes.io/service-account-token 3 15h
consul-consul-server-cert kubernetes.io/tls 2 15h
consul-consul-server-token-99nr6 kubernetes.io/service-account-token 3 15h
consul-consul-tls-init-token-7t52c kubernetes.io/service-account-token 3 11h
consul-tls kubernetes.io/tls 3 15h
default-token-s7mxx kubernetes.io/service-account-token 3 15h
sh.helm.release.v1.consul.v1 helm.sh/release.v1 1 15h
sh.helm.release.v1.consul.v2 helm.sh/release.v1 1 15h
sh.helm.release.v1.consul.v3 helm.sh/release.v1 1 11h
ui-tls kubernetes.io/tls 3 15h
``````
I changed consul config of Vault a bit but still no luck.
1)
```
storage "consul" {
path = "vault/"
address = "consul-consul-server.consul.svc.cluster.local:8501"
scheme = "https"
redirect_addr = "http://vault.vault.svc.cluster.local:8200"
VAULT_ADDR = "http://vault.vault.svc.cluster.local:8200"
token = "5f7cd758-6300-483e-blabla-3be6d889601e"
tls_ca_file = "/vault/userconfig/consul-ca-cert/tls.crt"
tls_cert_file = "/vault/userconfig/consul-consul-server-cert/tls.crt"
tls_key_file = "/vault/userconfig/consul-consul-consul-key/tls.key"
}
```
`Readiness probe failed: Error checking seal status: Get "http://127.0.0.1:8200/v1/sys/seal-status": dial tcp 127.0.0.1:8200: connect: connection refused`
2)
`kubectl create secret generic consul-client-ca --from-literal=ca="$(kubectl exec consul-consul-server-0 -n consul -- curl -sk https://localhost:8501/v1/connect/ca/roots | jq -r .Roots[0].RootCert)" -n vault`
and
```
storage "consul" {
path = "vault/"
address = "consul-consul-server.consul.svc.cluster.local:8501"
scheme = "https"
redirect_addr = "http://vault.vault.svc.cluster.local:8200"
VAULT_ADDR = "http://vault.vault.svc.cluster.local:8200"
token = "5f7cd758-6300-483e-blabla-3be6d889601e"
tls_ca_file = "/vault/userconfig/consul-client-ca/ca"
tls_cert_file = "/vault/userconfig/consul-consul-ca-cert/tls.crt"
tls_key_file = "/vault/userconfig/consul-consul-ca-key/tls.key"
}
```
`Readiness probe failed: Error checking seal status: Get "http://127.0.0.1:8200/v1/sys/seal-status": dial tcp 127.0.0.1:8200: connect: connection refused`
Which certs should I use?
What am I missing?
thanks&Regards
@aram
I think the solution is here .
it’s a really big adventure. I don’t know people reluctant to accomplish so complex venture.
if there is no any other reasonable solution/way for about 19 months; against to the documentation, we may say…
"Vault cannot connect to tls enabled Consul securely?
Reagrds
Could you please advise how to connect tls enabled Consul from Vault securely by a reasonable way?
Thanks & Regards
Hey @tirelibirefe
The answer would depend on whether you’re enabling autoEncrypt. If you are, then you need to first one you’ve mentioned. If not, then you need the second one.