Numerical index when for_each'ing a list of maps

Terraform
1

My goal is to build aws_network_acl_rule’s based on a list of maps which comes from a data source. I need a unique numerical index for each rule number. Here is the data source (as seen by terraform state show data.aws_ec2_managed_prefix_list.s3):

data "aws_ec2_managed_prefix_list" "s3" {
    address_family = "IPv4"
    arn            = "arn:aws:ec2:us-east-1:aws:prefix-list/pl-63a5400a"
    entries        = [
        {
            cidr        = "16.182.0.0/16"
            description = ""
        },
        {
            cidr        = "18.34.0.0/19"
            description = ""
        },
        {
            cidr        = "18.34.232.0/21"
            description = ""
        },
        {
            cidr        = "3.5.0.0/19"
            description = ""
        },
        {
            cidr        = "52.216.0.0/15"
            description = ""
        },
        {
            cidr        = "54.231.0.0/16"
            description = ""
        },
    ]
    id             = "pl-63a5400a"
    max_entries    = 0
    name           = "com.amazonaws.us-east-1.s3"
    owner_id       = "AWS"
    tags           = {}
    version        = 0
}

I’m already able to loop over the entries and get the CIDR’s for each rule like this:

resource "aws_network_acl_rule" "OutboundPrivateSubnetsToS3" {
  for_each   = {
    for index, ip in data.aws_ec2_managed_prefix_list.s3.entries : ip.cidr => ip
  }
  rule_number    = ???
  cidr_block     = each.value.cidr
  ...
}

How can I get a unique numerical index for each of those maps in the data source?

I’ve tried using rule_number = each.key, but that evaluates to rule_number = "16.182.0.0/16" and so on. I need something which evaluates to rule_number = 1 for the first rule, and rule_number = 2 for the second rule, and so on, with a unique integer for each entry.

Thanks.

2

Hi @jrobison-sb,

In order to have access to the index when configuring the arguments for your instances of this resource, you’ll need to include the value of the index symbol in your for expression as part of the data structure returned by that expression.

Here’s one way to do that:

  for_each = {
    for index, ip in data.aws_ec2_managed_prefix_list.s3.entries : ip.cidr => {
      ip_addr     = ip
      rule_number = index + 1
    }
  }

With for_each defined in that way, you can use each.value.ip_addr to access the ip object, and use each.value.rule_number to access the rule number, which I’ve defined here as being one greater than the index since you described the first item having number 1, while Terraform indexes lists using zero as the first element.

If you use this pattern, keep in mind that if you change the order of the elements in data.aws_ec2_managed_prefix_list.s3.entries then that may have significant impact on which rule numbers are assigned to each object, causing Terraform to plan lots of changes. That might be acceptable for what you are trying to achieve, but I mention it just in case it’s a concern.

Another caveat here is that my answer above assumes that data.aws_ec2_managed_prefix_list.s3.entries is a list, but the type of that value is actually decided by the provider as part of the schema for aws_ec2_managed_prefix_list, and the provider might consider that attribute to have a set type rather than a list type, in which case it will not preserve the order in which you wrote the objects. If that is true here then the assigned rule_number values will be arbitrary, since Terraform does not guarantee any particular iteration order for the elements of a set of objects.

3

@apparentlymart thanks for your super quick reply. You are always really helpful in this forum, I appreciate it.

I tried to implement your suggestion but couldn’t get it working. Here is a quickly reproducible paste with all the resources needed to reproduce this:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}

data "aws_ec2_managed_prefix_list" "s3" {
  name = "com.amazonaws.us-east-1.s3"
}

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_network_acl" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name = "main"
  }
}

resource "aws_network_acl_rule" "bar" {
  for_each = {
    for index, ip in data.aws_ec2_managed_prefix_list.s3.entries : ip.cidr => {
      ip_addr     = ip
      rule_number = index + 1
    }
  }
  network_acl_id = aws_network_acl.main.id
  rule_number    = each.value.rule_number
  egress         = true
  protocol       = "tcp"
  rule_action    = "allow"
  cidr_block     = each.value.ip_addr
  from_port      = 443
  to_port        = 443
}

Here is the error:

$ terraform plan

│ Error: Invalid operand
│ 
│   on main.tf line 34, in resource "aws_network_acl_rule" "bar":
│   34:       rule_number = index + 1
│ 
│ Unsuitable value for left operand: number required.

Any thoughts on how to unblock that?

And here are my versions, in case it matters:

$ terraform version
Terraform v1.6.6
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.32.1

Thanks.

4

Hi @jrobison-sb,

This error seems to suggest that index isn’t a number, which I think implies the possibility I was alluding to in my previous response: that this entries attribute is not actually a list of objects, and so there is no index to use here. (I suggested that it would work but produce an unspecified order, but I was forgetting that this was using index in a way that requires it to be a number, and so this error is what I should’ve expected in retrospect.)

That seems to be confirmed by the schema definition in the provider source code:

This is a set of objects, and therefore there is no defined order to these objects, and therefore no numeric indices to use.

That means that if you want to assign a numeric index to each one you will need to explain to Terraform how it should choose one based on the information that is available. I’m not sure what order would be appropriate for your situation, though. Do you have an idea for what would be a good rule for deciding what order to assign the rule numbers in, based only on the cidr attribute value?

5

@apparentlymart thanks for following up.

I don’t have strong feelings about how each cidr/rule should be ordered. I guess my slight preference would probably be:

  1. Whatever approach uses the cleanest and most readable code
  2. The same order that AWS returns them to the data source
  3. Just sort() them by cidr. High to low, or low to high, doesn’t matter.

FWIW I checked the aws_ec2_managed_prefix_list data source in three separate AWS accounts (same region), and I also checked aws ec2 get-managed-prefix-list-entries --prefix-list-id pl-63a5400a --region us-east-1 in a fourth account, and I got back the same CIDR’s in the same order across all four tries. So if these entries are ordered by anything already, they’re probably ordered by AWS.

Does that help explain the desired here? I’m happy to answer anything else if needed.

Thanks again.

6

If the order isn’t significant then one straightforward answer would be to convert this set to a list using the tolist function. That will return a list of objects whose order is not specified, and is subject to change between versions of Terraform, but should be consistent within a single specific version of Terraform.

Since my previous answer was assuming that this collection was already a list it should work to just introduce tolist in to what I previously suggested, like this:

  for_each = {
    for index, ip in tolist(data.aws_ec2_managed_prefix_list.s3.entries) : ip.cidr => {
      ip_addr     = ip
      rule_number = index + 1
    }
  }

My previous caution still applies, though: since the iteration order of a set of objects is not guaranteed by Terraform, upgrading to a new Terraform version might cause your existing objects to get their rule_number arguments reassigned due to the items no longer being in the same order.

Using a lexical sort of the cidr addresses would at least make the order guaranteed to remain stable across Terraform versions. That requires a slightly longer incantation:

  for_each = {
    for index, cidr in sort(data.aws_ec2_managed_prefix_list.s3.entries[*].cidr) : cidr => {
      cidr        = cidr
      rule_number = index + 1
    }
  }

The shape of this result is a little different: the cidr block is now just each.value.cidr rather than each.value.ip_addr.cidr as with my previous answers, because the splat operator ([*].cidr) is plucking out just the cidr attribute from each set element, which then causes the sort function to get a collection of strings rather than a collection of objects.

7

@apparentlymart thanks for your reply.

I had never seen that pattern before where you specify more than one element to the right of the => in a for_each, so this is super helpful.

Thanks again.

8

I’m glad it helped!

Just in case it’s an aid to learning, I would note that what I showed does still have only one expression to the right of the => symbol, but that expression is constructing a value of an object type.

Any single expression is allowed in that position, but some expression types have other expressions nested inside them. This particular example is of an object constructor, which builds an object-typed value based on the results of some other nested expressions.