Vault fails to start when using Godaddy certificate

Vault
1

I am trying to get the Hashicorp Vault UI to use HTTPS. I have a certificate from Godaddy which works on the same machine in apache2.

My vault.hcl file looks as follows

# HTTPS listener
listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_cert_file = "/godaddy_certs/123xxx321.crt"
  tls_key_file = "/godaddy_certs/privatekey.key"
  tls_disable = "false"
}

However I read here that I cannot simply use the certificate that was provided by Godaddy. That reference says the following:

To configure the listener to use a CA certificate, concatenate the primary certificate and the CA certificate together. The primary certificate should appear first in the combined file.

Now I am assuming that the “primary certificate” referred to here is the 123xxx321.crt file that was provided from Godaddy. Godaddy also include a gd_bundle-g2-g1.crt file.

So I thought I could just create a new file called myVaultCert.crt and copy the PEM string from 123xxx321.crt which looks like this:

-----BEGIN CERTIFICATE-----
MIIG ... /0I=
-----END CERTIFICATE-----

into the top position and then afterwards copy the PEM string from the gd_bundle-g2-g1.crt file.

So myVaultCert.crt now looks something like

-----BEGIN CERTIFICATE-----
MIIG36F ... /0I=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEADC ... v08=
-----END CERTIFICATE-----

I change my vault.hcl config to look as follows:

# HTTPS listener
listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_cert_file = "/godaddy_certs/myVaultCert.crt"
  tls_key_file = "/godaddy_certs/privatekey.key"
  tls_disable = "false"
}

When I run sudo systemctl start vault.service I get the following returned:

Job for vault.service failed because the control process exited with error code. See “systemctl status vault.service” and “journalctl -xe” for details.

When I check journalctl -xe I see this

Error initializing listener of type tcp: error loading TLS cert: decoded PEM is blank

So I went to Godaddy and saw there is a repository with links to root certificates and bundles etc etc. I have tried to copy the certificate that was provided folowed by several Certificates found in that repository but they all give me the same error.

What certificates must I concatenate so that I can use a Godaddy certificate with Hashicorp Vault?

2

You have the right idea. You simply concatenate the certificate issued to your domain name, with the certificate(s) of the relevant intermediate CAs, forming a chain connecting back to the relevant root CA.

By the way, this is exactly the same format that the Apache HTTPD consumes too. You should be able to use exactly the same files you’re referencing in your Apache HTTPD configuration file, with Vault.

I’ve never seen

before. It makes me think you’ve done something unusual/unexpected in forming the “combined” certificate file.

Please identify which GoDaddy certificate issued your Vault certificate, and show us here. You can use the openssl command line tool to do this. For example, here I am using it to show the issuer of my Exim mailserver’s certificate:

$ openssl x509 -noout -issuer -in /etc/exim4/fullchain.pem 
issuer=C = US, O = Let's Encrypt, CN = R3
3

Thank you. This is good to know.

I ran the command below openssl x509 -noout -issuer -in /godaddy_certs/123xxx321.crt

and this was returned.

issuer=C = US, ST = Arizona, L = Scottsdale, O = “GoDaddy.com, Inc.”, OU = Repository, CN = Go Daddy Secure Certificate Authority - G2

When I run that same command against my “combined” pem file I get the exact same response.

to create the “combined” file I simply ran cat 123xxx321.crt gd_bundle-g2-g1.crt > myCert.crt

Not sure what else to try anymore.

4

That much is expected, as the openssl x509 command only reads the first certificate found in the input.

Hmm… Sounds fine… Would it be possible for you to attach the resulting myCert.crt so I can directly see the content that Vault is rejecting?

5

Sure,

Thanks for the help.

myCert.crt.txt (7.1 KB)

6

Hmm… this all looks fine. The only thing of possible note is that the file is using Windows line endings - CRLF rather than LF. I suppose that might be the issue - worth a quick test to see if it is.

Just as a minor efficiency note, the 4th certificate in this file is redundant - as it is a self-signed root certificate, so cannot link trust from one CA to another. Including it is not harmful, but does make every TLS connection setup slightly longer for no benefit.

7

Having done a GitHub code search for "decoded PEM is blank", it looks like it was a false path to be looking in to the certificate file at all. It appears this error means something is wrong with the private key file.

8

Yeah apologies I had to create the file on a windows system not where the actual file was.

I can download the actual file if that will help. But I think your later post has more clarify.

Also thanks for the heads up on the 4th certificate.

9

This is really interesting.

I can upload the privatekey as well … I just won’t use this certificate and will re-key it later if this will help ro identify the problem.

10

Thank you VERY much … You pointed me in the right direction. I removed the privatekey file and copied the text from the generated-private-key.txt from godaddy. I pasted the test into a file in the ubuntu system making sure there were no ^m line endings and it is now working.

Great catch about the "decoded PEM is blank", it looks like it was a false path to be looking in to the certificate file at all. It appears this error means something is wrong with the private key file. This put me on the correct path.

Fixing the privatekey fixed the problem.