Set a list attribute with results from file()

Terraform
1

Suppose I have a string delimited, comma separated text file containing lines like these:

"s3:CreateBucket",
"s3:DeleteBucket",
"s3:DeleteBucketWebsite"

I’d like to pass those as input to a module’s attribute which expects a list of strings. In this example, consider the “actions” block in a statement in an aws_iam_policy_document.

If I set that list as a literal string, it works fine:

locals { actions = { s3_write =["s3:CreateBucket", "s3:DeleteBucket",        "s3:DeleteBucketWebsite"]

data "aws_iam_policy_document" "document" {
  statement {
        effect    = "Allow"
        resources = ["*"]
        actions   = local.actions["s3_write"]
      }
  }

However, if I change the local variable to read from the text file with the file() function, I get a variety of MalformedPolicyExceptions when I apply the plan. I have tried many iterations to parse the file into a list which matches what the actions attribute of statement expects:

s3_write = file("${path.module}/policies/s3_write.txt")
s3_write = tolist("${path.module}/policies/s3_write.txt")
s3_write = split(",", file("${path.module}/policies/s3_write.txt"))

I know this is subtle. Any help would be appreciated. Thanks.

2

With the input file:

"s3:CreateBucket",
"s3:DeleteBucket",
"s3:DeleteBucketWebsite"

$ terraform console
> flatten(regexall("\"(.*)\"", file("./file")))
[
  "s3:CreateBucket",
  "s3:DeleteBucket",
  "s3:DeleteBucketWebsite",
]
>

If the format can be changed, the expression can be simplified by leaving out the quotes and commas. file() outputs a string, and doesn’t know about comma delimiting.

s3:CreateBucket
s3:DeleteBucket
s3:DeleteBucketWebsite

$ terraform console
> split("\n", chomp(file("file2")))
[
  "s3:CreateBucket",
  "s3:DeleteBucket",
  "s3:DeleteBucketWebsite",
]
1 Like
3

Thanks for those suggestions, @jeremykatz!

I just wanted to add the general observation here that whenever you read data from a file or similar location that stores “sequence of bytes” you will always need to do some kind of parsing.

Terraform automatically interprets the sequence of bytes as UTF-8 to convert the sequence of bytes into a Terraform string, which is a sequence of unicode characters.

If you want something other than a string then you’ll need to parse further. Since you’ve described what is essentially a custom file format here you will need to design your own way to parse it, and @jeremykatz has some good suggestions for that and also implied the other thing I wanted to call out specifically: if the input format is under your control then you can potentially tailor it to make the result easier to consume using Terraform’s existing parsing functions.

For example, if you can arrange for the input to be formatted as a JSON array then you could parse it with jsondecode, or if it can be formatted as CSV then you could parse it with csvdecode.

Terraform has regexall, split etc because sometimes you just need to extract data from an odd, non-standard format and these serve as a pragmatic way to get that done. I’d always recommend using a standard encoding format where possible though, because the resulting Terraform configuration will be a simpler statement of your intent (“intepret this string as JSON!”) and the parsers built in to Terraform are generally able to give better errors if the input isn’t formatted correctly, whereas a custom format interpreted using lower-level functions may respond to invalid input either with a confusing error or with no error at all and just some strange behavior.