Unable to export service account key in Terraform #GCP

Terraform
1

I have downloaded the GCP service account key to my local system. In Terraform, I have set the GOOGLE_APPLICATION_CREDENTIALS as a path to this file in the startup-script part of my bastion instance. Below is a snippet:

variable “credentials”{
default=“C:/GCP/service-account-key.json”
}
.
.
metadata= {
startup-script=<<SCRIPT
export GOOGLE_APPLICATION_CREDENTIALS="{file("{var.credentials}")}"
SCRIPT
}
.
.

Later I have written a #!/bin/bash script to store this credentials to another file as below:

#!/bin/bash
.
.
.
#creating credentials file
printf “$GOOGLE_APPLICATION_CREDENTIALS” > /home/ubuntu/credentials
.
.
.
But when I open the above credentials file, the file is truncated as below and the entire key is missing:

{
type: service_account,
project_id: acn-devopsgcp,
private_key_id: xxxxx,
private_key: -----BEGIN

Can please someone let me know why the service account key is not getting exported properly to the file or if there is anything that needs to be corrected.

EDIT: the code snippet is attached in the file for further reference. test-file.txt (1000 Bytes)

2

I’m guessing it’s because the service account key contains quotes and new line characters that need to be escaped.
I would suggest base64 encoding the service account key when writing to the startup-script, and base64 decoding when you write to file at the far end:
printf "$GOOGLE_APPLICATION_CREDENTIALS" | base64 -d > /home/ubuntu/credentials

However, instead of manually handling a service account key, a better option might be to associate the Service Account with the VM, so applications can automatically authenticate with the GCP API:

https://cloud.google.com/docs/authentication/production#obtaining_credentials_on_compute_engine_kubernetes_engine_app_engine_flexible_environment_and_cloud_functions

https://cloud.google.com/compute/docs/access/service-accounts#associating_a_service_account_to_an_instance

You can specify the Service Account via Terraform:

3

Thanks for the suggestion. I will try out the changes.
I still need the GOOGLE_APPLICATION_CREDENTIALS value exported for my application use case.
In this sense, I have another approach. I have the GCP service account key file in JSON format. Is there a way in Terraform to provide the “contents” of a JSON file directly (instead of specifying the path to the file) in a Terraform variable block and then have Terraform interpret is as JSON?
I have seen that Terraform has jsonencode and jsondecode functions, but not able to find many examples on it.
Is there any other way to do this?
Below is the approach I’m looking at:

variable “credentials”{
type = “string”
default="<<<.contents of key file in JSON format.>>>"
}

In Bastion start up script:

#!/bin/bash
export GOOGLE_APPLICATION_CREDENTIALS= jsonencode("${file(var.credentials)}")

EDIT: I tried the base64 method suggested, but its giving out the below error and hence not exporting the key file correctly. The key file is blank.
startup-script: INFO startup-script: base64: invalid input

4

Hi,

To store the full json key as a string in a Terraform variable, you can use the heredoc syntax:

variable "jsonkey" {
  default = <<EOF
{
  "this": "is",
  "json": {
    "inline": true
  }
}
EOF
}

I don’t think you need to use the jsondecode when you populate the GOOGLE_APPLICATION_CREDENTIALS env var, just let Terraform pass the json string directly.
But make sure you wrap it in quotes:

export GOOGLE_APPLICATION_CREDENTIALS="${var.jsonkey}"
1 Like
5

Hi,

Thanks for the help. I have declared the variable this way:

variable “jsonkey” {
default = <<EOF
{
“type”: “service_account”,
“project_id”: “xxxx”,
“private_key_id”: “xxxx”,
“private_key”: “-----BEGIN PRIVATE KEY-----\nXXXXXXXXXXX\n-----END PRIVATE KEY-----\n”,
“client_email”: "xxx@xxxx.iam.gserviceaccount.com",
“client_id”: “xxxx”,
“auth_uri”: “xxxx”,
“token_uri”: “xxxx”,
“auth_provider_x509_cert_url”: “xxxx”,
“client_x509_cert_url”: “xxxx”,
“json”: {
“inline”: true
}
}
EOF
}

Then, in my start up script, I have exported the variable as per your suggestion and then I have copied this data to a file:
#!/bin/bash
export GOOGLE_APPLICATION_CREDENTIALS="${var.jsonkey}"
printf “$GOOGLE_APPLICATION_CREDENTIALS” > /home/ubuntu/.gcp/credentials

But the credentials file has the below data:
{
type: service_account,
project_id: xxxx,
private_key_id: xxxxxx,
private_key: -----BEGIN

The entire key isn’t exported to the file. I even tried the base64 encoding-decoding as below:
export GOOGLE_APPLICATION_CREDENTIALS="${var.jsonkey}" | base64
printf “$GOOGLE_APPLICATION_CREDENTIALS” | base64 -d > /home/ubuntu/.gcp/credentials
But this time, the credentils file is totally blank.

So I have tried another approach in my startup script:

#!/bin/bash
cat < /etc/gce_creds.json
“${var.jsonkey}”
EOF

export GOOGLE_APPLICATION_CREDENTIALS=$(cat /etc/gce_creds.json | base64)
printf “$GOOGLE_APPLICATION_CREDENTIALS” | base64 -d > /home/ubuntu/.gcp/credentials

Here, the credentials file has the following data:
{
“type”: “service_account”,
“project_id”: “xxxxx”,
“private_key_id”: “xxxx”,
“private_key”: “-----BEGIN PRIVATE KEY-----\nXXXXXXXXXXX\n-----END PRIVATE KEY-----\n”,
“client_email”: "xxx@xxxxx.iam.gserviceaccount.com",
“client_id”: “xxxx”,
“auth_uri”: “xxxx”,
“token_uri”: “xxxx”,
“auth_provider_x509_cert_url”: “xxxx”,
“client_x509_cert_url”: “xxxx”,
“json”: {
“inline”: true
}
}

Ideally, the credentials file should not contain the last few lines:
“json”: {
“inline”: true
}

Is there anything that can be done to avoid this and get a proper file?

6

There was an extra space and a comma inserted added at the end of the file, by removing which I was able to get the correct file.

7

I have a question regarding the inclusion of a start-up script as a file,
metadata_startup_script = “${file(“startup-script.sh”)}”
If I declare a variable in the terraform script how the script can access ${var.jsonkey} if the script is a reference to a file??

8

how this works if your startup script is a $file call from terraform and not inline script?

9

Generally , this is how we use a variable with reference to a file

test_var = "${file(var.var_name)}"

variable "var_name" {
  default = "<path_to_the_file>"
}