Failing to use HCP Consul as my terraform backend

HashiCorp Cloud Platform (HCP)
1

I am struggling to use HCP Consul as my terraform backend. I run terraform from my Ubuntu laptop. I have enabled public endpoint for my cluster consul-cluster-01. This is what I have tried:

  1. I downloaded the client config from https://portal.cloud.hashicorp.com/services/consul/consul-cluster-01?project_id=<pid>
    This is a ZIP file with ca.pem.

  2. I then used this for my “terraform init” and get error:

[INFO]  Terraform version: 0.15.3
[INFO]  Go runtime version: go1.16.2
[INFO]  CLI args: []string{"/home/…/terraform", "init", "--backend-config=access_token=…", "--backend-config=datacenter=consul-cluster-01", "--backend-config=address=https://consul-cluster-01.consul.<pid>.aws.hashicorp.cloud", "--backend-config=scheme=https", "--backend-config=path=terraform_backend/dev04/terraform.tfstate", "--backend-config=ca_file=/home/…/client_config_bundle_consul_consul-cluster-01/bundle/ca.pem"}

│ Error: Error inspecting states in the "consul" backend:
│     Get "https://consul-cluster-01.consul.<pid>.aws.hashicorp.cloud/v1/kv/terraform_backend/dev04/terraform.tfstate-env:?dc=consul-cluster-01&keys=&separator=%2F": x509: certificate signed by unknown authority
  1. curl works properly (does not throw certificate error):
> curl --head 'https://consul-cluster-01.consul.<pid>.aws.hashicorp.cloud/ui/consul-cluster-01/services'

HTTP/2 200 
accept-ranges: bytes
content-type: text/html; charset=utf-8
date: Mon, 12 Jul 2021 23:58:52 GMT
last-modified: Fri, 09 Jul 2021 23:47:11 GMT
strict-transport-security: max-age=31536000
content-length: 8317
2

Issue resolved with help of HCP support. This was the input that helped:

Don't include the "--backend-config=ca_file=/home/…/client_config_bundle_consul_consul-cluster-01/bundle/ca.pem"

I was able to reproduce this error by including the "--backend-config=ca_file" and it was successful when i removed the variable.

ca_file is only needed for Consul Agent Communication (https://learn.hashicorp.com/tutorials/consul/tls-encryption-secure)

The API call we are making here is encrypted by Let's Encrypt, and we do not need to specify the ca_file. The settings as described here (https://www.terraform.io/docs/language/settings/backends/configuration.html#partial-configuration) is all you need.