The "for_each" value depends on resource attributes that cannot be determined until apply

Terraform
1

As so many people before me (and most probably after me), I ran into the following dreaded problem:

The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.

I did some digging and I must admit that no explanation really gives full insights into what the actual problem is. I thought I saw someone mentioning somewhere that for_each can’t rely on variable parameters, which seems a bit silly as that is maybe one of the only reasons it’s useful to use (or at least a very important one).

I’ll illustrate with an example. In my code I’m creating a number of service accounts:

resource "google_service_account" "service_account_one" {
  project     = var.project_id
  account_id  = "sa-one"
}

resource "google_service_account" "service_account_two" {
  project     = var.project_id
  account_id  = "sa-two"
}

Now I want to give these service accounts a set of roles on particular objects. For example, granting them access to a storage bucket. Instead of copy/pasting the same resource 3 times, I thought I’d be smart and solve it with a locals list.

locals {
  service_account_emails = [
    google_service_account.service_account_one.email,
    google_service_account.service_account_two.email,
  ]
}

And then give them the desired IAM role:

resource "google_storage_bucket_iam_member" "configuration_access" {
  for_each = toset(local.service_account_emails)
  bucket   = google_storage_bucket.configuration_bucket.name
  member   = "serviceAccount:${each.value}"
  role     = "roles/storage.admin"
}

But, as you already guessed:

â•·
│ Error: Invalid for_each argument
│ 
│   on ../../../main.tf, in resource "google_storage_bucket_iam_member" "configuration_access":
│  345:   for_each = toset(local.service_account_emails)
│     ├────────────────
│     │ local.service_account_emails is tuple with 2 elements
│ 
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.
╵

So, can someone shed some light on what is happening in the background here? As you can see, it’s a fixed list with 2 elements, so I have no idea why Terraform can’t predict how many elements are going to be created.

This isn’t the first time I’m running into this problem. I have other examples where it’s problematic as well, especially with flattened lists and passing in variables into a module.

Is there work underway to solve this problem or at least give a better error message and guidance on how to solve this?

1 Like
2

Hi @debakkerbjorn,

The problem here is not that terraform cannot determine the length of the list, in fact you could use count = length(local.service_account_emails) and it should work just fine. The problem is that the values of the list are unknown, so there is no way to determine what the index key for each of your resource instances is going to be.

The simplest resolution here, since you already are statically defining the structure of the values, is to use a map with predefined keys rather than a set.

locals {
  service_account_emails = {
    one = google_service_account.service_account_one.email,
    two = google_service_account.service_account_two.email,
  }
}
1 Like
3

Wanted to say that this solution saved me a lot of headache trying to solve this for_each problem. Changing it to a map with the keys did indeed satisfy Terraform to ignore the “unknown” number of objects generated from my other modules I was passing in. Thank you!

4

Rather than manually declaring keys, you can do something like this

resource "google_storage_bucket_iam_member" "configuration_access" {
  for_each = {for i, val in local.service_account_emails: i => val}
  bucket   = google_storage_bucket.configuration_bucket.name
  member   = "serviceAccount:${each.value}"
  role     = "roles/storage.admin"
}

Honestly, I think the many, many issues I see after Googling this error are largely caused by the suggestion in the documentation to use toset(). Converting your list to a map instead of a set is simply better.

I’ve put my concern in another discussion here, though honestly I don’t know if this is the right forum for it.

2 Likes
5

@jkemsley,

While valid, the drawback to your approach is that you are directly coupling the resource instances to the order of the values in service_account_emails. This means that any change to the order will destroy and recreate the resources, and adding or removing any values from the list will destroy and recreate all values later in the list. What you have is exactly equivalent to count, only formatting the indexes as strings:

resource "google_storage_bucket_iam_member" "configuration_access" {
  count =  length(local.service_account_emails)
  bucket   = google_storage_bucket.configuration_bucket.name
  member   = "serviceAccount:${local.service_account_emails[count.index]}"
  role     = "roles/storage.admin"
}
6

Yep, you’re right on that. I was mistaken on what was happening inside for_each. Thanks for catching that

7

@jbardin - Can you please elaborate on this statement?

The problem is that the values of the list are unknown

In my variables.tf I have a defined a required set(string) …

variable "vpc_endpoints" {
  description = "IDs for this VPC endpoints"
  type        = set(string)
}

And when I try to use this with a for_each, I get the error under discussion here…

│   40:   for_each = var.vpc_endpoints
│     ├────────────────
│     │ var.vpc_endpoints is set of string with 1 element
│ 
│ The "for_each" set includes values derived from resource attributes that cannot be determined ...

I’m providing a static set here, how can it be that these values are “unknown” ? But if I define this as a map(string) (and give it some bogus, fixed key that I don’t care about) the values are “known”? Why are the values known if it’s a map but not if it’s a set ?

I guess it’s a bit weird that when you use for_each you get a thing called “each” that has a key and a value even when you’re iterating over a collection of single values. I’ve tried making my variable a set(string) and using each.key and each.value and both yield the same failure. So… is the issue that somewhere Terraform has to derive a “key” for each “value” in the set ?

8

Yes, this is the crux of it. The for_each feature is not only used for providing multiple values to a configuration block, but it’s there to derive a unique address for each instance for Terraform to track and for expressions elsewhere in the configuration to reference.

1 Like