For_each on a count resource and unknown keys: alternatives vs -target?

Terraform
1

Hi:

I’m trying to create an AWS load balancer with target group attachments, exactly like the documentation here shows (under “Registering Multiple Targets”):
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_target_group_attachment

My code differs only slightly in that the resources are created in a module, and I was using toset() rather than a k,v strategy, so the corresponding code looks like:

resource "aws_instance" "worker-nodes" {
  ami           = data.aws_ami.generic.id
  instance_type = var.worker_node_type

  count = var.worker_node_count
...

and

resource "aws_lb_target_group_attachment" "main" {
  for_each = toset(module.app-infra.worker-nodes.*.id)

  target_group_arn = module.loadbalancer.lb_target_group.arn
  target_id = each.key
  port = 80
}

This actually worked fine as I built up my configuration step-wise and then added the attachments after the worker nodes were up and running. Now when I change the worker nodes definition such that they require recreation, it fails because the for_each can’t know worker-nodes.*.id until later. It fails of course for the exact same reason when using the map instead of set, as shown in the example documentation.

The error message from terraform is in this case reasonably helpful and although I understand it, I include it here for completeness:

│ Error: Invalid for_each argument
│
│   on main.tf line 81, in resource "aws_lb_target_group_attachment" "main":
│   81:   for_each = toset(module.app-infra.worker-nodes.*.id)
│     ├────────────────
│     │ module.app-infra.worker-nodes is tuple with 2 elements
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so Terraform
│ cannot determine the full set of keys that will identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your
│ configuration and where only the values contain apply-time results.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends
│ on, and then apply a second time to fully converge.

Q1: Shouldn’t the documentation be updated to reflect that the example shown in fact does not work?

Q2a: Is there an alternative strategy to create statically-known key names – for example, stringifying the actual terraform resource name? I looked for some function to do this but came up empty-handed. I envisioned a map something like:

{
  "module.app-infra.worker-nodes[0]" => "...id determined dynamically from .id"
...
}

Q2b: Or, is there an an entirely different way to be doing this?

Q3: If no to Q2a/2b, is the only solution multiple passes using -target ?

Thanks in advance!

2

Hi @jblachly,

The problem arises mainly from the use of the id attribute in particular, since for EC2 instances that is an unpredictable string chosen by the remote API during create.

If you use a different attribute of aws_instance that you’ve set directly in the configuration then that would work, although EC2 instances in particular don’t have any system-enforced unique key chosen by the user, so you’d need to invent your own scheme here such as using an element of tags which you make sure is unique across all of the instances.

Using for_each downstream of something that’s using count is a little strange, though. count typically implies that all of the instances are fungible – that they are all equal and there are only more than one for redundancy. But to then try to assign each one a unique name would suggest that they aren’t as fungible as they were implied to be.

Normally I would recommend being consistent throughout a family of related resources: either also using count for the target group attachments (correlating only by the automatically-assigned indices) or using for_each on both and starting with a map of objects data structure that names each instance with a unique key and describes what’s unique about each one. Which approach to choose depends on whether you consider these instances to be fungible or not.

1 Like
3

Using for_each downstream of something that’s using count is a little strange, though. count typically implies that all of the instances are fungible – that they are all equal and there are only more than one for redundancy. But to then try to assign each one a unique name would suggest that they aren’t as fungible as they were implied to be.

Disagree with conclusion: these EC2 instances are fungible; the only reason one needs a “name” (or id, really) is for assignment to the load balancer. Iterating with for_each over an arbitrary collection seems natural.

Normally I would recommend being consistent throughout a family of related resources: either also using count for the target group attachments (correlating only by the automatically-assigned indices)

This is excellent advice and probably the best solution in this case. I had gotten away from using count indices for anything because of the possibility of re-ordering triggering things. I still use count for fleets of EC2 instances, however.