Terrraform For each example

Terraform
1

Hi,

I have the following code

data "aws_nat_gateway" "shoot_nat_gateway_z0" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "test-natgw-z0"
 }
}

data "aws_nat_gateway" "shoot_nat_gateway_z1" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "test-natgw-z1"
 }
}

data "aws_nat_gateway" "shoot_nat_gateway_z2" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "test-natgw-z2"
 }
}

How can i simplify with a for each loop assuming i count the length of the cidr block list ?

output "subnet_cidr_block" {
  value = [for s in data.aws_subnet.shoot_vpc_subnet : s.cidr_block]
}

How i can generate the value to test-natgw-z0, test-natgw-z1, test-natgw-z2 … on the fly in terraform ?

2

Hi @linuxbsdfreak,

I’m not sure I’m following your example correctly because your first block shows use of a data.aws_vpcs while your second shows data.aws_subnet and so I’m not sure how the two are connected. However, I think you are asking how to create one NAT gateway per VPC id in data.aws_vpcs.shoot_vpc_id.ids, so I’m going to try to answer that here.

My initial instinct would be to do this using resource for_each where the VPC id is the unique key, which is convenient because the ids attribute is already a set of strings and so we can just pass it directly to for_each as long as that data source is able to read during the planning step:

data "aws_nat_gateway" "shoot" {
  for_each = data.aws_vpcs.shoot_vpc_id.ids

  vpc_id = each.key
  tags = {
    "Name" = "test-natgw-${each.key}"
  }
}

This would declare instances like aws_nat_gateway.shoot["vpc-abc123"], assuming you have a VPC with id vpc-abc123. However, it doesn’t exactly match what you showed in your example because the tagged names here would be like test-natgw-vpc-abc123. The advantage of this strategy is that if you add a new VPC later then Terraform can just create the new NAT gateway for it without disturbing any of the existing ones.

If you want to assign them names with incrementing integers like you showed in your example then that is possible but will come with an important consequence: if you add a new VPC later which has an id that sorts earlier than one of the existing ones then all of the subsequent VPCs will have their tag names updated to represent their new positions in the sequence, which feels conceptually weird given that VPCs are not really an ordered data type.

With that said, to get that done you’d need to transform that set of id strings into a data structure that has both a name and an index for each one. I’m going to assume we still want Terraform to track the VPCs by their remote ids, and so produce a map from VPC id to index:

data "aws_nat_gateway" "shoot" {
  for_each = { for i, id in sort(data.aws_vpcs.shoot_vpc_id.ids) : id => i }

  vpc_id = each.key
  tags = {
    "Name" = "test-natgw-${each.value}"
  }
}

Because the for_each expression now produces a map from VPC id string to index, each.key inside this block is the VPC id and each.value is the assigned index. Terraform will still track these objects by their associated VPC id, but will generate the Name tags based on their position in the sort order.

If you use this variant then I’d encourage you to experiment with applying once with an initial set of VPC ids, and then changing your set of VPC ids by adding and removing elements before running terraform apply again, and make sure you’re comfortable with the plan Terraform makes in those scenarios, because it’s better to understand the consequences of adding and removing VPCs from your set during development than to get caught out by it once your module is already in production.

3

Hi @apparentlymart,

Thanks for the reply. I have the following file

data "aws_vpcs" "shoot_vpc_id" {
  tags = {
    Name = var.shoot_name
  }
}

data "aws_subnet_ids" "shoot_vpc_subnets_ids" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)

  tags = {
    "kubernetes.io/role/elb" = "use"
  }
}

data "aws_subnet" "shoot_vpc_subnet" {
  for_each = data.aws_subnet_ids.shoot_vpc_subnets_ids.ids
  id       = each.value
}

data "aws_nat_gateway" "shoot_nat_gateway_z0" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "shoot--test--development-natgw-z0"
 }
}

data "aws_nat_gateway" "shoot_nat_gateway_z1" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "shoot--test--development-natgw-z1"
 }
}

data "aws_nat_gateway" "shoot_nat_gateway_z2" {
  vpc_id = element(tolist(data.aws_vpcs.shoot_vpc_id.ids), 0)
  tags = {
    "Name" = "shoot--test--development-natgw-z2"
 }
}


output "vpc_id" {
  value = data.aws_vpcs.shoot_vpc_id.ids
  description = "Shoot VPC Id"
}

output "subnet_id" {
  value = [for s in data.aws_subnet.shoot_vpc_subnet : s.id]
  description = "Shoot Subnet Ids"
}

output "subnet_availability_zone" {
  value = [for s in data.aws_subnet.shoot_vpc_subnet : s.availability_zone]
  description = "Shoot Subnet Availibility Zones"
}

output "subnet_cidr_block" {
  value = [for s in data.aws_subnet.shoot_vpc_subnet : s.cidr_block]
  description = "Shoot Subnet CIDR Block"
}


output "natgateway_public_ip_az0" {
  value = data.aws_nat_gateway.shoot_nat_gateway_z0.public_ip
  description = "Shoot NAT Public IP AZ0"
}

output "natgateway_public_ip_az1" {
  value = data.aws_nat_gateway.shoot_nat_gateway_z1.public_ip
  description = "Shoot NAT Public IP AZ1"
}

output "natgateway_public_ip_az2" {
  value = data.aws_nat_gateway.shoot_nat_gateway_z2.public_ip
  description = "Shoot NAT Public IP AZ2"
}

As you see i am trying to get the external public IPs attached to the NAT GW. The problem is that the VPC may have 1,2 or 3 NatGWs attached in the VPC and i would like to get the info dynamically regardless of how may NAT GWs are created in VPC.

I am searching on tags

shoot--test--development-natgw-z* This part i can take store it as a variable shoot–test–development-natgw- . The issue is that would like to loopover irrespective of the NAT GWs.

Kevin

4

Hi @apparentlymart,

The issue with the data is that it cannot get multiple resources with the same key value on the aws tags. It gives an error with to add additional filters. The requirement is to display the public ips of the natgw Dynamically irrespective of how many are created or else I have to do some sort of if else in the template

Thanks once again .
Kevin

5

Hi @linuxbsdfreak,

That particular requirement doesn’t seem to be possible with the current capabilities of the AWS provider: I don’t see a data source there that would allow querying for multiple NAT gateways at once based on filtering criteria.

It seems like there is an underlying API to query NAT gateways by filters, so there could potentially be an aws_nat_gateway_ids data source that would behave similarly to aws_subnet_ids, but I guess so far nobody has needed that enough to implement it.

I think my best suggestion at this point would be to open a feature request issue for this in the AWS provider repository, and if the provider team is open to it perhaps also open a pull request for it, using the other VPC-related plural data sources as an example to build from – most of them follow a similar pattern.

One workaround that did come to my mind, just from looking at how all of these AWS objects are connected, is that if your NAT gateways are created one-to-one with subnets then you could potentially use your existing data.aws_subnet.shoot_vpc_subnet as the for_each basis and then query individual NAT gateways by subnet ID:

data "aws_nat_gateway" "default" {
  for_each = data.aws_subnet.shoot_vpc_subnet

  subnet_id = each.value.id
}

Of course, that won’t work if you don’t have exactly one NAT gateway per subnet, but if your NAT gateways are somehow else created systematically based on the presence of some other collection of objects then perhaps you could take a similar strategy with those objects as the for_each.

My knowledge of AWS in particular is rusty, but someone in the AWS provider community forum might have a more imaginitive idea.

6

Hi @apparentlymart . It works after doing the changes