Terraform for_each "will be known only after apply" workaround not working for multiple entries

Terraform
1

Hello Terraform community,

I am trying to workaround the known limitations with terraform for_each arguments by using a Map instead of a Set. I have found this suggested and have done this several times already but currently I am facing the problem that my workaround seems fine with a single Set element but does not work with multiple elements.

This is the HCL code I am using.

variable "allowed_groups" {
  type        = set(string)
}

locals {
  allowed_groups_map = { for idx, val in tolist(var.allowed_groups) : idx => val }
}

resource "test_resrouce" {
  for_each   = length(local.allowed_groups_map) != 0 ? local.allowed_groups_map : map("0", data.okta_group.everyone.id)
}

I created Outputs to debug this behaviour:

# 1 String item in the given Set
allowed_groups_map = {
   + "0" = (known after apply)
}

# 2 string items in the given Set
allowed_groups_map = (known after apply)

What I would have expected and I do not understand why I do not get this result with 2 Set elements:

allowed_groups_map = {
   + "0" = (known after apply)
   + "1" = (known after apply)
}

Does anybody know what I am doing wrong here?
Or is this actual a bug/known behaviour?

Any input appreciated!
Thanks

2

It seems like the problem is related to the function call in the for loop. It works fine with a simple list like:

allowed_groups_map = { for idx, val in tolist(var.allowed_groups) : idx => val }
3

Hi @dhohengassner,

What’s going on here is a bit subtle.

You have defined your input variable as being a set of strings. An important characteristic of sets is that the values inside them are unique, so if asked to create a set with two equal elements Terraform will coalesce them both into a single element and therefore produce a set with fewer elements than it was constructed from.

When dealing with values that won’t be known until the apply step Terraform has to approximate as best as possible the results of any operation, which includes the operation of converting a list into a set. Terraform must not “over-promise” because downstream expressions will make assumptions based on the unknown results, and so might for example produce an incorrect plan.

If you were to assign to your input variable a sequence with known string values like the following…

allowed_groups_map = ["a", "b"]

… this would produce a set with two elements.

However, if you assigned this one…

allowed_groups_map = ["a", "a"]

… that would produce a set with only one element.

If Terraform doesn’t yet know what values those strings will have then it can’t determine whether the result will have one or two elements, and so it needs to be conservative and only promise that the result will be a set, without any prediction about the length of that set.

When you use only a one-element list Terraform can tell that can’t coalesce because there is nothing for it to coalesce with, so in that case Terraform can confidently predict that the result will be a list with one element even though it doesn’t yet know what that element is.

A common answer to this sort of problem is to use a map instead of a set and make sure that all of the map keys are known even though some of the map values may be unknown. When for_each is a map only the keys decide which instance addresses are being declared, so the values in the map are irrelevant and can be unknown or known with equivalent behavior.

1 Like
4

Thanks @apparentlymart for that explanation! That sounds reasonable and explains that behaviour :+1:

Adjusted the input value to a list and checked on the validation rule if the elements are unique.