Embedded interpolations

Terraform
1

Is there a way to do embedded interpolations?

This doesn’t work, but goal is something like this "${aws_s3_bucket.${each.value}.arn". I’ve tried "${format("aws_s3_bucket.%s.arn", "${each.value}")}", but it doesn’t interpolate the outer ${}.

My goal is to create a bunch of bucket policies for a list of buckets. The policies will only differ by the bucket name.

data "aws_iam_policy_document" "s3_transport" {
  for_each = local.s3_buckets_set

  statement {
    sid    = "DenyNonsecureTransport"
    effect = "Deny"
    principals {
      type        = "*"
      identifiers = ["*"]
    }
    actions = ["s3:*"]
    resources = [
      "${format("aws_s3_bucket.%s.arn", "${each.value}")}",
      "${format("aws_s3_bucket.%s.arn", "${each.value}")}/*"
    ]
    condition {
      test     = "Bool"
      variable = "aws:SecureTransport"
      values   = [false]
    }
  }
}

And then reference them like below:

data "aws_iam_policy_document" "bucket1" {
  source_policy_documents = [data.aws_iam_policy_document.s3_transport["bucket1"].json]

data "aws_iam_policy_document" "bucket2" {
  source_policy_documents = [data.aws_iam_policy_document.s3_transport["bucket2"].json]

The above bucket1 and bucket2 policies will have other local policy statements that differ.

Maybe there is a better way to do this?

2

Hi @Kimmel,

Terraform’s template syntax is for constructing data (strings) rather than code. There is no way to use Terraform code to generate more Terraform code to be evaluated dynamically, because Terraform uses a data-flow-oriented model where it needs to understand the dependency relationships between all objects before it can evaluate any expressions.

However, from your example it seems like local.s3_buckets_set is a dynamic collection of S3 buckets and so in principle you could declare all of your S3 buckets with the same resource using for_each over that same collection:

resource "aws_s3_bucket" "example" {
  for_each = local.s3_buckets_set

  # ...
}

Dependencies are between resources rather than resource instances in Terraform, so you can dynamically refer to an instance of a resource as long as you can statically specify which resource it belongs to. For a resource with for_each set, the resource appears in expressions elsewhere as a map using the same keys, so you can chain resources together:

resource "aws_iam_policy_document" "s3_transport" {
  for_each = aws_s3_bucket.example

  statement {
    # ...
    resources = [
      each.value.arn,
      "${each.value.arn}/*",
    ]
    # ...
  }
}

Notice that because for_each in this case is the other resource rather than the original local value, each.value here refers to the aws_s3_bucket.example objects themselves, and so each.value.arn is the ARN of the bucket whose key is the same as the current policy document instance key.

As long as the differences between the objects are systematic enough that you can describe them as part of local.s3_buckets_set (which you will probably need to turn into a map of objects whose values describe the differences, if it’s currently as set as the name suggests), you can hopefully handle this whole problem via chained for_each in this way, and thus avoid the need to explicitly declare multiple instances of any resource.

However, in situations where the resources are not systematic at all and so for_each isn’t helpful, you can get a similar effect by constructing your own map of objects, which achieves the same effect as for_each would but allowing you to choose for yourself which objects to include, regardless of how they were declared:

locals {
  s3_buckets_set = toset(["a", "b"])

  s3_buckets = {
    a = aws_s3_bucket.a
    b = aws_s3_bucket.b
  }
}

You can then use local.s3_buckets in a similar way as I used the for_each chaining in the earlier example, since it’s a map of objects just like aws_s3_object.example was in that previous case:

resource "aws_iam_policy_document" "s3_transport" {
  for_each = local.s3_buckets

  statement {
    # ...
    resources = [
      each.value.arn,
      "${each.value.arn}/*",
    ]
    # ...
  }
}

Notice that what we’re doing here is looking up attributes/elements in collections using normal expressions, rather than using code to generate other code. That’s the crucial difference here: Terraform can understand all of the relationships prior to evaluating any expressions in this case, because all of the code is visible in the first pass and there’s no situation where a string in the program could be reinterpreted as code later on.

1 Like
3

@apparentlymart ,

Truly appreciate the reply.

Unfortunately the (numerous) buckets already exist, and I’d prefer to not have to move things around in state so I will be using your second example.

It appears to do exactly what I need. When/if you have time I have a few questions:

  • I understand the discussion of resourcing chain and frequently use for_each chaining. However, I’m still not 100% on why the original format didn’t work. Is it simply because format results in a string which cannot be interpolated? I.e., is it really passing ${"aws_s3_bucket.a.arn"} instead of ${aws_s3_bucket.a.arn}?

  • In your s3_buckets map you have a = aws_s3_bucket.a. I understand placing quotes around the object name results in a string - which we don’t want. Does placing the key in quotes "a" = aws_s3_bucket.a change anything?

5

The format function can only ever produce a string, and Terraform will never parse and evaluate that string to look for more variables. Using format in that way is really just the same as template interpolation: it’s just a different syntax for the same result.

For an object constructor expression in particular, because literal attribute names are the far more common case Terraform will understand a naked identifier like that a as being a literal attribute name rather than a reference. That is a special case for object construction and isn’t true in any other situations.