Trying to create a secret string

Terraform
1

Hi,
I’m trying to populate aws secrets with a common values map(string) and using templatefile to expand the values. The problem is that I can’t find a way to make it work. If I use jsondecode, it complains of the parameter not being a string. This is the code line

secret_string=jsonencode(merge(each.value.secrets, jsondecode(try(templatefile("${path.module}/secret_templates/${each.key}.tftpl",local.common_secrets), {}))))

If I use a for loop to manually parse the templatefile result, I get a bunch of extra quotes and \n in the output:

processed = { for line in split(",",chomp(try(templatefile("${path.root}/template.tftpl", var.common_secrets),{}))):
split(":", chomp(line))[0] =>split(":", line)[1]
}

an example of the tftpl file is:

{
"FIREBASE_PROJECT_ID" : "${FIREBASE_PROJECT_ID}",
"FIREBASE_DATABASE_URL" : "${FIREBASE_DATABASE_URL}",
"FIREBASE_ADMIN_private_key_id" : "${FIREBASE_ADMIN_private_key_id}",
"FIREBASE_ADMIN_private_key" : "${FIREBASE_ADMIN_private_key}",
"FIREBASE_ADMIN_client_email" : "${FIREBASE_ADMIN_client_email}",
"FIREBASE_ADMIN_client_id" : "${FIREBASE_ADMIN_client_id}",
"FIREBASE_DEFAULT_STORAGE_BUCKET" : "${FIREBASE_STORAGE_BUCKET}",
"SIMPLE_SERVER_KEY" : "",
"SIMPLE_SERVER_DOMAIN" : "",
"MAIL_DEFAULT_OUTBOUND_EMAIL" : "${MAIL_DEFAULT_OUTBOUND_EMAIL}",
"MAIL_SMTP_PORT" : "${MAIL_SMTP_PORT}",
"MAIL_SMTP_HOST" : "${MAIL_SMTP_HOST}",
"MAIL_SMTP_USER" : "${MAIL_SMTP_USER}",
"MAIL_SMTP_PASSWORD" : "${MAIL_SMTP_PASSWORD}",
"NODE_ENV" : "${ENV}"}

and the values (all fake):

variable "common_secrets" {
type=map(string)
default={
ENV="production"
FIREBASE_PROJECT_ID="test-tf"
FIREBASE_API_KEY="AIzaSsfwesdfsrsdfT6RWuCiuX3SGJs"
FIREBASE_DEFAULT_STORAGE_BUCKET="test-tf.appspot.com"
FIREBASE_STORAGE_BUCKET="test-tf.appspot.com"
FIREBASE_STORAGE_BUCKET_ASIA="test-tf.appspot.com"
FIREBASE_STORAGE_BUCKET_EUROPE="test-tf.appspot.com"
FIREBASE_appId="1:3451341341:web:123412341234"
FIREBASE_AUTH_DOMAIN="test-tf.firebaseapp.com"
FIREBASE_DATABASE_URL="https://test-tf.firebaseio.com"
FIREBASE_messagingSenderId="1234123412"
FIREBASE_ADMIN_auth_uri="https://accounts.google.com/o/oauth2/auth"
FIREBASE_measurementId="G-1234123412"
FIREBASE_ADMIN_client_email="1341234f60ic@test-tf.iam.gserviceaccount.com"
FIREBASE_ADMIN_private_key="-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkq1234123412341243FQcA+4N2ia9IA4sUsbxF7gZIxMcg12341234\nMvk234234234234234t399\nKwJHpvEgy5HkMlD23423423OGYeN\nyP23c9q/er+\nhG6vINcUM/re+vXyTA\re+LE9d9v1JR4Pi\nL2z/rerw34+543/5gQNbbFJiedPs82ZVXa7\n54/34\nsWNjfwOxMJ2z0BVdAID/n+rzy5FCz98qq\nLGH8r4LGZwypDHdXsoOQOsIHI+4\nKhVJ3xyiVnkWvs+4+4+g9Iw+4/4\nyXeKGZHkbtDvwvm/4/ouaYH5QZQNBsD+i+GyBmKtll\n/4/hID3dEJUk+4\njXXBVH/4/uyK38iu5czNo47Qyjyjmrfa+eLvG\n+4/QGUlBm\n5HLSyS9xdHK1v9r8ArvjrqfMJcqP6rha+4\n17Ut/4/rhMVp+3\n/Ws/ZAoGAEpV+2\n+YBZnmd2Y4hvkhzpmZO1rFEGGXBsqwJHiecZyse+wdUca06uEFRV5yBjQKzCUfVW\n9T0Oh63b42G4g/3\nFzxuiIEWn1Nf2VmAHvlMXpw=\n-----END PRIVATE KEY-----\n"
FIREBASE_ADMIN_private_key_id="1234123412341"
FIREBASE_ADMIN_client_id="123412341234"
FIREBASE_ADMIN_token_uri="https://oauth2.googleapis.com/token"
FIREBASE_ADMIN_auth_provider_x509_cert_url="https://www.googleapis.com/oauth2/v1/certs"
FIREBASE_ADMIN_client_x509_cert_url="https://www.googleapis.com/robot/v1/metadata/x509/iam.gserviceaccount.com"
FIREBASE_ADMIN_type=""
TWILIO_ACCOUNT_SID="134234234234"
TWILIO_API_KEY="234234234"
TWILIO_API_SECRET="234234234"
TWILIO_AUTH_TOKEN="234234234"
APP_GATEWAY_URL="https://gw-stg-aws.test.com"
MAIN_APP_URL="https://stg-aws.test.com"
MAIN_ST_APP_URL="https://st-stg-aws.test.com"
HQ_APP_URL="https://hq-stg-aws.test.com"
GATEWAY_APP_URL="https://gw-stg-aws.test.com"
INTERNAL_SERVER_URL="https://int-stg-aws.test.com"
DISCOVER_SERVER_URL="https://dvr-stg-aws.test.com"
SESSION_SECRET="4234234234234234"
REQUEST_KEY_FOR_ADMINS="234234234234"
TOPIA_SERVICE_TOKEN="234234234234234234"
NPM_TOKEN="234234234234234234234"
MC_SERVER_URL="https://test.com"
MC_INSTANCE_ID="234234234234"
MC_AUTHORIZATION_KEY="234234234234"
MAIL_DEFAULT_OUTBOUND_EMAIL="outbound@mg.test.com"
MAIL_SMTP_PORT="465"
MAIL_SMTP_HOST="smtp.mailgun.org"
MAIL_SMTP_USER="outbound@mg.test.com"
MAIL_SMTP_PASSWORD="2342342342-234234-23423"
}
 
}

Output for the for loop:

output = {
  <<-EOT
  
  "FIREBASE_ADMIN_client_email" 
  EOT = " \"1341234f60ic@test-tf.iam.gserviceaccount.com\""
  <<-EOT
  
  "FIREBASE_ADMIN_client_id" 
  EOT = " \"123412341234\""
  <<-EOT
  
  "FIREBASE_ADMIN_private_key" 
  EOT = <<-EOT
   "-----BEGIN PRIVATE KEY-----
  MIIEvQIBADANBgkq1234123412341243FQcA+4N2ia9IA4sUsbxF7gZIxMcg12341234
  Mvk234234234234234t399
  KwJHpvEgy5HkMlD23423423OGYeN
  yP23c9q/er+
e+LE9d9v1JR4Pi+vXyTA
  L2z/rerw34+543/5gQNbbFJiedPs82ZVXa7
  54/34
  sWNjfwOxMJ2z0BVdAID/n+rzy5FCz98qq
  LGH8r4LGZwypDHdXsoOQOsIHI+4
  KhVJ3xyiVnkWvs+4+4+g9Iw+4/4
  yXeKGZHkbtDvwvm/4/ouaYH5QZQNBsD+i+GyBmKtll
  /4/hID3dEJUk+4
  jXXBVH/4/uyK38iu5czNo47Qyjyjmrfa+eLvG
  +4/QGUlBm
  5HLSyS9xdHK1v9r8ArvjrqfMJcqP6rha+4
  17Ut/4/rhMVp+3
  /Ws/ZAoGAEpV+2
  +YBZnmd2Y4hvkhzpmZO1rFEGGXBsqwJHiecZyse+wdUca06uEFRV5yBjQKzCUfVW
  9T0Oh63b42G4g/3
  FzxuiIEWn1Nf2VmAHvlMXpw=
  -----END PRIVATE KEY-----
  "
  EOT
  <<-EOT
  
  "FIREBASE_ADMIN_private_key_id" 
  EOT = " \"1234123412341\""
  <<-EOT
  
  "FIREBASE_DATABASE_URL" 
  EOT = " \"https"
  <<-EOT
  
  "FIREBASE_DEFAULT_STORAGE_BUCKET" 
  EOT = " \"test-tf.appspot.com\""
  <<-EOT
  
  "MAIL_DEFAULT_OUTBOUND_EMAIL" 
  EOT = " \"outbound@mg.test.com\""
  <<-EOT
  
  "MAIL_SMTP_HOST" 
  EOT = " \"smtp.mailgun.org\""
  <<-EOT
  
  "MAIL_SMTP_PASSWORD" 
  EOT = " \"2342342342-234234-23423\""
  <<-EOT
  
  "MAIL_SMTP_PORT" 
  EOT = " \"465\""
  <<-EOT
  
  "MAIL_SMTP_USER" 
  EOT = " \"outbound@mg.test.com\""
  <<-EOT
  
  "NODE_ENV" 
  EOT = " \"production\"}"
  <<-EOT
  
  "SIMPLE_SERVER_DOMAIN" 
  EOT = " \"\""
  <<-EOT
  
  "SIMPLE_SERVER_KEY" 
  EOT = " \"\""
  <<-EOT
  {
  "FIREBASE_PROJECT_ID" 
  EOT = " \"test-tf\""
}

For the jsondecode:

│ Error: Invalid function argument
│ 
│   on modules/ecs/secrets.tf line 14, in resource "aws_secretsmanager_secret_version" "secrets_version":
│   14:   secret_string = jsonencode(merge(each.value.secrets, jsondecode(try(templatefile("${path.module}/secret_templates/${each.key}.tftpl",local.common_secrets), {}))))
│     ├────────────────
│     │ while calling jsondecode(str)
│     │ each.key is "test"
│     │ path.module is "test/test"
│
2

Hi @notchjpl,

You seem to have missed the actual error message part of the error output you shared! The most important part is the text immediately after the source code snippet, since that part typically gives the details about what was invalid about the argument.

Based entirely on the surrounding context, I suspect the jsondecode function is returning a JSON syntax error. Although I can’t be sure if it’s true in this place with the information given, invalid JSON syntax is a very common outcome of trying to generate JSON using string concatenation (which is what Terraform template interpolation does), which is why the templatefile documentation recommends against doing that (and offers an alternative).

3

Sorry I missed your reply as I was expecting an email notification on a reply.
I will try to solve the issue as pointed, but as this is a map of strings, not sure it will fit the example (still need to figure that).
Will reply here after

4

@apparentlymart I finally decided to use local variables to build the structures. The for loops suggested in the article you pointed were not good enough to allow me creating the different contents, so I had to hardcode the templates as locals.
Wouldn’t it be great to enable using external scripting like invoking python or any other language that may be used to create more complex code passed to terraform?
Regards

5

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.