How to create resource based on other resources dynamically

Terraform
1

Hi Everyone

I am trying to do the following

  1. Create subnets from a data structure (based as input variable)
  2. Then create VPC endpoints (Gateway Load balancer Endpoints) for each subnet from above

Target Environment = AWS Terraform Version = latest

Module - subnet

data "aws_availability_zones" "available" {}

locals {
 all_subnets = flatten([
 for type, details in var.subnets_cidr_details : [
 for cidr in details.cidr : {
   type  = type
   index = index(details.cidr, cidr)
   cidr  = cidr
 }
 ]
 ])
}

resource "aws_subnet" "subnets" {
 for_each = {
 for subnet in local.all_subnets : subnet.cidr => subnet
 }
 vpc_id = var.vpc_id
 cidr_block = each.value.cidr
 availability_zone = data.aws_availability_zones.available.names[each.value.index]
 tags = {
   Name = "${var.service_name}-${each.value.type}-${regex("..$", data.aws_availability_zones.available.names[each.value.index])}"
   Service = each.value.type
 }

Module - endpoints

resource "aws_vpc_endpoint" "main" {
  for_each          = var.endpoint_subnets.id
  service_name      = var.endpoint_service_name
  subnet_ids        = [each.key]
  vpc_endpoint_type = "GatewayLoadBalancer"
  vpc_id            = var.vpc_id
}

The main.tf looks like this

module "subnets" {
  depends_on = [
  module.vpc_main]
  source               = "./modules/subnets"
  service_name         = var.service_name
  subnets_cidr_details = var.subnets_cidr_details
  vpc_id               = module.vpc_main.vpc_id
}

module "endpoints" {
  depends_on = [
  module.endpointservice, module.subnets]
  source = "./modules/endpoints"
  endpoint_service_name = module.endpointservice.endpoint_service_name
  vpc_id = module.vpc_main.vpc_id
  endpoint_subntes =  [for x in module.subnets.subnets_info: x.id if x.tags_all["Service"] == "endpoints"]
  service_name = var.service_name
}

terraform.autovars

subnets_cidr_details = {
  endpoints = {
    cidr = ["10.0.0.11/28", "10.0.0.12/28", "10.0.0.13/28"]
  },
  public = {
    cidr = ["10.1.2.120/28", "10.1.2.140/28", "10.1.2.150/28"]
  }
}

When i run this, I get the following error “var.endpoints_subnets.id will be known only after apply
│
│ The “for_each” value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to
│ first apply only the resources that the for_each depends on.”

I understand that for_each does not for values that will be known after apply . But how do i create other resources like nats, endpoints based on number of subnets ?

One way I found is if i create the subnet and the endpoints in same module it works

For example

locals {
  endpoint_subnets = flatten([
    for type, details in var.endpoint_cidrs : [
      for cidr in details.cidr : {
        type  = type
        index = index(details.cidr, cidr)
        cidr  = cidr
      }
    ]
  ])
}


resource "aws_subnet" "endpoint_subnets" {
  for_each = {
    for subnet in local.endpoint_subnets : subnet.cidr => subnet
  }
  vpc_id = var.vpc_id
  cidr_block = each.value.cidr
  availability_zone = data.aws_availability_zones.available.names[each.value.index]
  tags = {
    Name = "${var.service_name}-endpoint-${each.value.type}-${regex("..$", data.aws_availability_zones.available.names[each.value.index])}"
  }
}


resource "aws_vpc_endpoint" "main" {
  for_each          = aws_subnet.endpoint_subnets
  service_name      = var.endpoint_service_name
  subnet_ids        = [each.key]
  vpc_endpoint_type = "GatewayLoadBalancer"
  vpc_id            = var.vpc_id
}

This works however as you see , I need to provide separate input for each type of subnet I want to create.

2

I’d offload this to an existing module or check how it is done there.

3

Hi @abhip,

In order to correlate objects between the configuration, the prior state, and the plan Terraform needs each resource instance to have a stable address, which for for_each includes the keys from whatever map you pass to the for_each argument.

Because those keys need to remain static throughout the lifecycle of the object (including before it has been created), Terraform requires that you use values that are decided statically inside your configuration rather than values that are decided dynamically after the object has been created.

Since AWS subnet IDs are server-decided and not predictable before creation, they are not suitable to use as instance identifiers. Instead, you must place the dynamically-chosen subnet IDs only in the value of the for_each map, making sure that the keys are all statically-defined.

With the fragments you’ve shared here I’m unfortunately not sure how these different parts are connected in order to give a concrete example, but I will describe a general design pattern and then hopefully you will see how to make use of it in your specific case.

The root idea is to have the original data structure (an input variable, in your case) include a key to use to identify the corresponding objects. A common way to achieve that is to define the variable as taking a map of objects, where the map keys will also serve as the instance keys for the declared objects:

variable "subnets" {
  type = map(
    object({
      cidr_block        = string
      availability_zone = string
    })
  )
}

module "network" {
  source = "../modules/network"

  subnets = var.subnets
}

When defining the value for this variable, you’d choose a suitable identifying key for each of the subnets. This can be anything you like, so you might choose to name it something that’s meaningful to your specific use-case. In this case I chose “public” and “private”, but that’s just an example and you can use any key names that will make them all unique.

subnets = {
  public = {
    cidr_block        = "10.1.0.0/17"
    availability_zone = "us-east-1a"
  }
  private = {
    cidr_block        = "10.1.127.0/17"
    availability_zone = "us-east-1a"
  }
}

With that defined, the important pattern is to preserve those keys throughout all other work your configuration does. For example, if you have a module which takes that description of the subnets, declares them using an aws_subnet resource, and returns the IDs then the important rule would be to make the output value also be a map, and use the same keys as given in the variable:

variable "subnets" {
  type = map(
    object({
      cidr_block        = string
      availability_zone = string
    })
  )
}

resource "aws_subnet" "example" {
  for_each = var.subnets

  cidr_block        = each.value.cidr_block
  availability_zone = each.value.availability_zone
}

output "subnets" {
  value = {
    for k, s in aws_subnet.example : k => {
      id                = s.id
      cidr_block        = s.cidr_block
      availability_zone = s.availability_zone
    }
  }
}

Now the calling module can refer to that subnets output in its own for_each, or can pass that map on to some other module that will use it in that way, and the original keys "public" and "private" will be preserved throughout, which will avoid the error message you saw because the dynamically-chosen values (the id values of the subnets, in this case) only ever appear in the value of the map, where they therefore won’t be used as part of any resource instance addresses.

The subnets declared in my examples above would be assigned Terraform addresses like this, where the instance key is set to the key originally provided, and not to any dynamically-chosen values about the objects:

  • module.network.aws_subnet.example["public"]
  • module.network.aws_subnet.example["private"]