Creating a TC Workspace Variable using API whose value is a JSON document

Terraform HCP Terraform
1

Hi,
I’m trying to create an environment variable in my Terraform Cloud Workspace that contains a GCP Service Account Key document. This is a JSON document which includes an attribute named private key, which consists of a PEM encoded private key (i.e. ----BEGIN PRIVATE KEY----…), however the newline characters have been replaced with a backslash n sequence of characters.

From the User Interface (app.terraform.io), I couldn’t paste in the document as-is. There was a message that indicated that Variable values can’t include newline characters. So, I simply removed all newline characters (cat credentials.json | tr -d ‘\n’) and was able to successfully paste the output through the UI and make use of it.

My problem is that I’m now trying to achieve the same result from a script using curl and the workspaces/:workspace_id/vars endpoint. Depending on the escaping strategy that I try I either get an HTTP 400 or an HTTP 422 response. I finally got the API to accept the value by preceding every double quote character (") in the value by a backslash (i.e. sed ‘s/"/\"/g’) and by replacing every sequence of backslash n (\n) with double backslash n (i.e. sed ‘s/\n/\\n/g’). However, this value is now unusable, because when I try to execute a Terraform run that relies on that environment variable to configure the credentials of the google provider, it complains that the private key is unreadable; that it must be either a PEM or a plain PKCS1 or PKCS8.

If this were a Terraform variable, I would simply base64 encode the lot and use the base64decode function in my Terraform configuration, but the value needs to be consumable as-is by the google Terraform provider.

Obviously the UI on app.terraform.io knows how to escape this value in order to apply it successfully. Any pointers would be greatly appreciated.

Thanks,
Marc

2

After a couple of showers, I came up with an idea to debug this issue and found the solution. The reason I couldn’t see what my error was that I was marking the variable as sensitive, which it should be. But, I temporarily created the variable without marking it sensitive using the UI and then another one using my script. This allowed me to compare the two values and see what I was doing wrong.

In my script, I was piping the credentials file to tr -d '\n ', to remove both newline characters and spaces (I was only thinking about the JSON). But that also affected the PEM encoded private key, as it transformed the header and footer from -----BEGIN PRIVATE KEY----- to -----BEGINPRIVATEKEY-----. By changing the -d argument in the tr command to \n and then ensuring that the output from that entire pipeline was surrounded with double quotes, the variable value was correct.