Create subnets from list with object

Terraform
1

Hi,

I’m stucked with the following subnets configuration:

  + aws = [
      + {
          + accounts        = "prod"
          + id              = "523131231043"
          + private_subnets = {
              + eu-central-1a = "10.44.4.96/27"
              + eu-central-1b = "10.44.5.128/27"
              + eu-central-1c = "10.44.6.160/27"
            }
          + public_subnets  = {
              + eu-central-1a = "10.44.7.0/27"
              + eu-central-1b = "10.44.8.32/27"
              + eu-central-1c = "10.44.9.64/27"
            }
        },
      + {
          + accounts        = "dev"
          + id              = "098453041777"
          + private_subnets = {
              + eu-central-1a = "10.44.7.0/27"
              + eu-central-1b = "10.44.8.32/27"
              + eu-central-1c = "10.44.9.64/27"
            }
          + public_subnets  = {
              + eu-central-1a = "10.44.10.96/27"
              + eu-central-1b = "10.44.12.160/27"
              + eu-central-1c = "10.44.12.160/27"
            }
        },
    ]

I’ve created the subnet resource this way:

resource "aws_subnet" "private" {
  count = length(var.aws[*].private_subnets)

  availability_zone = element(keys(var.aws[*].private_subnets), count.index)
  cidr_block        = element(values(var.aws[*].private_subnets), count.index)
  map_public_ip_on_launch = false
   vpc_id            = aws_vpc.this.id
}

This produces this error:

│ Error: Invalid function argument
│
│   on test.tf line 42, in resource "aws_subnet" "private":
│   42:   availability_zone = element(keys(var.aws[*].private_subnets), count.index)
│     ├────────────────
│     │ var.aws is tuple with 2 elements
│
│ Invalid value for "inputMap" parameter: must have map or object type.
╵
╷
│ Error: Invalid function argument
│
│   on test.tf line 42, in resource "aws_subnet" "private":
│   42:   availability_zone = element(keys(var.aws[*].private_subnets), count.index)
│     ├────────────────
│     │ var.aws is tuple with 2 elements
│
│ Invalid value for "inputMap" parameter: must have map or object type.
╵
╷
│ Error: Error in function call
│
│   on test.tf line 43, in resource "aws_subnet" "private":
│   43:   cidr_block        = element(values(var.aws[*].private_subnets), count.index)
│     ├────────────────
│     │ var.aws is tuple with 2 elements
│
│ Call to function "values" failed: values() requires a map as the first
│ argument.
╵
╷
│ Error: Error in function call
│
│   on test.tf line 43, in resource "aws_subnet" "private":
│   43:   cidr_block        = element(values(var.aws[*].private_subnets), count.index)
│     ├────────────────
│     │ var.aws is tuple with 2 elements
│
│ Call to function "values" failed: values() requires a map as the first
│ argument.
╵
ERRO[0012] 1 error occurred:
	* exit status 1

I have tried concating and a lot of other stuff but I don’t get this stuff to work. I need help please. I need to work with count as I need to create a NAT GW only in the first subnet of each account. I was able to work trough with for_each but then I’m not able to have a NAT GW only in the first subnet. It then creates them in all puplic subnets.

2

Hi @AlexanderWiechert,

var.aws[*].private_subnets would produce a value like the following:

[
  {
    eu-central-1a = "10.44.4.96/27"
    eu-central-1b = "10.44.5.128/27"
    eu-central-1c = "10.44.6.160/27"
  },
  {
    eu-central-1a = "10.44.7.0/27"
    eu-central-1b = "10.44.8.32/27"
    eu-central-1c = "10.44.9.64/27"
  },
]

…so this error message does seem to be correct, in that you’re asking for keys(...) of a value that isn’t a mapping and therefore doesn’t have any keys.

I’m not sure I fully follow what your goal was here; count = length(var.aws[*].private_subnets) would declare only two subnets, but I’d guess from the context that you probably intended to declare six private subnets so that there’s one for each distinct CIDR block in the expression above. I’m going to answer with that assumption in mind, and if I’ve misunderstood your goal then hopefully you can see how to get from this partial answer to something that gets the result you need.

The general idea here is that you’ll need an intermediate flattened data structure which has one element per instance you want to create. This is a specific example of Flattening nested structures for for_each, and so the answer here will follow the same principles as the one in that documentation.

I’m going to focus only on the private subnets here for brevity’s sake, but the same principle would apply to your public_subnets attribute and associated resource too.

locals {
  private_subnets = flatten([
    for account in var.aws : [
      for az, cidr in account.private_subnets : {
        account_name      = account.accounts
        availability_zone = az
        cidr_block        = cidr
      }
    ]
  ])
}

The above declares that local.private_subnets is a flat sequence of objects, with a structure like this:

[
  {
    account_name      = "prod"
    availability_zone = "eu-central-1a"
    cidr_block        = "10.44.4.96/27"
  },
  {
    account_name      = "prod"
    availability_zone = "eu-central-1b"
    cidr_block        = "10.44.5.128/27"
  },
  # ...
  {
    account_name      = "dev"
    availability_zone = "eu-central-1a"
    cidr_block        = "10.44.7.0/27"
  },
  # ...
]

I included the account_name argument in these objects because we need enough information in these objects to generate a unique identifier so that Terraform can track these subnets in the state to understand how their configurations have changed in future runs. The availabililty zone doesn’t seem sufficient because you’ll have two subnets in each AZ, and I typically wouldn’t use cidr_block as part of the unique key because that is a direct property of the object we’re declaring, rather than an identifier for some object it’ll be related to.

Finally we can now declare the subnets themselves:

resource "aws_subnet" "private" {
  for_each = {
    for s in local.private_subnets :
    "${s.account_name}:${s.availability_zone}" => s
  }

  availability_zone = each.value.availability_zone
  cidr_block        = each.value.cidr_block
  vpc_id            = aws_vpc.this.id

  map_public_ip_on_launch = false
}

I used the for_each argument instead of count here because these declared instances are materially different from one another, rather than just being “scaled out” interchangable copies of the same service. The for_each expression generates a map with keys based on the account name and availability zone, and so Terraform will track these subnets with addresses like this:

  • aws_subnet.private["prod:eu-central-1a"]
  • aws_subnet.private["prod:eu-central-1b"]
  • …
  • aws_subnet.private["dev:eu-central-1a"]
  • …

Tracking them in this way means that if you add a new entry to the top-level list or to any of the nested public_subnets mappings then Terraform will be able to understand that your intent is to add one or more new instances of this resource, without disturbing any of the existing ones. If you were to remove one of the entries from one of the private_subnets maps then Terraform will propose to destroy the corresponding subnet without interfering with the others.

3

Hi, that worked perfect. How do I ensure now the a NST GW is only setup in the first public subnet? That’s why I wanted to try with count as I’m able there to reference it via index 0. How do I do this with the for_each approach?

I mean is it possible to reference the first item in this loop? I tried this but this still creates them in all subnets.

resource "aws_nat_gateway" "this" {
  for_each = {
    for s in local.private_subnets : "${s.account_name}:${s.availability_zone}" => s
  }

  allocation_id = aws_eip.this.id
  subnet_id = aws_subnet.public[element([each.key], 0)].id


}
4

Hi @AlexanderWiechert,

You specified your subnets in a map, and map elements are unoredered so there isn’t really any inherent meaning of “first” there, but there are two possible approaches here:

  1. Add a new attribute alongside public_subnets to specify which AZ should host the NAT gateway. For example, you could specify nat_gateway_az = "eu-central-1a" and then use that to make your selection. This would be the most explicit approach.
  2. Decide on a systematic rule for deciding which AZ is “first”, such as taking the first one alphabetically. This would avoid adding a new attribute, but is less explicit and could potentially risk churn if you later introduced a new AZ that was alphabetically “earlier” than the previous selection. (Admittedly it seems unlikely that there would be any later AZ that would sort before eu-central-1a, though.)

Below the surface these options are basically the same thing, with the second case just moving the definition of nat_gateway_az into a local value encapsulated inside the module, describing it as a systematic rule rather than an explicit setting:

locals {
  # NOTE: This will fail if there isn't at least one private
  # subnet defined for each account.
  nat_gateway_azs = {
    for account in var.aws :
    account.accounts => keys(account.private_subnets)[0]
  }
}

Now, regardless of whether it’s explicitly specified or inferred automatically, you’ll have a direct lookup from account to its “nat gateway AZ”, which you can use in the definition of your resource:

resource "aws_nat_gateway" "this" {
  for_each = {
    for account in var.aws : account.accounts => account
  }

  subnet_id = aws_subnet.public["${each.key}:${each.value.nat_gateway_az}"].id

  # NOTE: I copied this verbatim from your example,
  # but I suspect you'll need to set for_each on
  # aws_eip.this too, and then use
  # aws_eip.this[each.key].id instead here.
  allocation_id = aws_eip.this.id
}

resource "aws_nat_gateway" "this" {
  for_each = {
    for account in var.aws : account.accounts => account
  }

  subnet_id = aws_subnet.public["${each.key}:${local.nat_gateway_azs[each.key]}"].id

  # (note as above)
  allocation_id = aws_eip.this.id
}

Notice that for this resource there is only one per account, so we don’t need the same compound keys that the subnets had. These will have addresses like aws_nat_gateway.this["prod"].

5

Hi @apparentlymart works perfect. Thanks a lot.

6

One last question. I know it is possible to enable resource creation via count but using count and for_each together does not work., How can I enable/disable creation of resources then?

7

count and for_each work the same way if you are wanting to enable/disable the whole resource creation. For count you need to use 0 to disable, and for for_each you need to have an empty map. If enabled you’d set count to the number of resources you want (e.g. 1) while for for_each you’d set it to the map you are wanting to use.

8

Do you have an example how this could look?

9

I don’t know if this was exactly what @stuart-c had in mind, but a way I typically do conditionals with for_each is to use the if clause of a for expression in such a way that it filters out all of the elements in the case where I want the resource to be effectively “disabled”:

resource "aws_nat_gateway" "this" {
  for_each = {
    for account in var.aws : account.accounts => account
    if var.manage_nat_gateway
  }

  # ...
}

While the if clause will often refer to one of the temporary symbols defined in the for expression (account in this case), it doesn’t necessarily need to; you can refer to anything in the module’s global scope if you want to make a more unilateral decision like the above, where it’s an “all or nothing” sort of situation.

If you need to be more fine-grain then you can of course combine a temporary-symbol-based expression with a global one:

  if var.manage_nat_gateway && account.enabled

(I know account.enabled wasn’t a real thing in your case, but just using that for a simple example of something that might be set on a per-element basis.)

10

That works, as does something like for_each = var.enabled ? var.aws : {}

11

A conditional expression can work too, but you need to take some extra care with types in that case, because Terraform’s conditional operator tries to infer a single result type for both the true and false arms so that it can still type-check even when the condition isn’t known yet. (We originally intended the conditional operator mainly for switching between primitive-typed values where the type conversions are more straightforward/intuitive; there’s some potentially-surprising behavior for collection types there as a legacy of the less sophisticated type system from Terraform v0.11 and earlier.)

I would typically suggest using the for expression because then there’s no question about what the result type will be; it will always be an object if using the { } brackets, and always a tuple if using the [ ] brackets.