For_each value depends on resource attributes that cannot be determined until apply

Terraform
1

I have a root module which calls child module with this call:

module "routing_extapp_data" {
    source = "../../modules/Stack/Routing"
    routing = { 
        src_rt_ids = module.extapp_vpc.private_route_table_ids
        dst_rt_ids = module.data_vpc.private_route_table_ids
        src_cidr = module.extapp_vpc.vpc.cidr_block
        dst_cidr = module.data_vpc.vpc.cidr_block
        peering_connection_id = module.peering_extapp_data.peering_connection_ids
        name = "extapp-data"
        src_provider = "aws.stack"
        dst_provider = "aws.stack"
    }
    providers = {
        aws.stack = aws.stack
        aws.ops = aws.ops
    }
}

In child module I have:

resource "aws_route" "src_stack" {
    timeouts {
        create = "5m"
        delete = "5m"
    }
    for_each = {for rt_id in var.routing.src_rt_ids: format("%s-%s", var.routing.name, rt_id) => rt_id if var.routing.src_provider == "aws.stack"}

    provider = aws.stack
    route_table_id = each.value
    destination_cidr_block = var.routing.dst_cidr
    vpc_peering_connection_id = var.routing.peering_connection_id

}

However it gives me this error:

The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.

Logs says:

2020/02/19 14:49:38 [TRACE] evalVariableValidations: not active for module.routing_extapp_intapp.var.routing, so skipping

2020/02/19 14:49:38 [TRACE] [walkPlan] Exiting eval tree: module.routing_extapp_intapp.var.routing

2020/02/19 14:49:38 [TRACE] vertex “module.routing_extapp_intapp.var.routing”: visit complete

2020/02/19 14:49:38 [TRACE] dag/walk: visiting “module.routing_extapp_intapp.aws_route.dst_stack”

2020/02/19 14:49:38 [TRACE] vertex “module.routing_extapp_intapp.aws_route.dst_stack”: starting visit (*terraform.NodePlannableResource)

2020/02/19 14:49:38 [TRACE] vertex “module.routing_extapp_intapp.aws_route.dst_stack”: evaluating

2020/02/19 14:49:38 [TRACE] [walkPlan] Entering eval tree: module.routing_extapp_intapp.aws_route.dst_stack

2020/02/19 14:49:38 [TRACE] module.routing_extapp_intapp: eval: *terraform.EvalWriteResourceState

2020/02/19 14:49:38 [WARN] Provider “Terraform Registry” produced an invalid plan for module.peering_extapp_intapp.aws_vpc_peering_connection_accepter.stack_accept_peering[0], but we are tolerating it because it is using the legacy plugin SDK.

The following problems may be the cause of any confusing errors from downstream operations:

  • .accepter: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead

  • .requester: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead

2020/02/19 14:49:38 [TRACE] module.peering_extapp_intapp: eval: *terraform.EvalCheckPreventDestroy

2020/02/19 14:49:38 [TRACE] module.peering_extapp_intapp: eval: *terraform.EvalWriteState

2020/02/19 14:49:38 [TRACE] EvalWriteState: writing current state object for module.peering_extapp_intapp.aws_vpc_peering_connection_accepter.stack_accept_peering[0]

2020/02/19 14:49:38 [ERROR] module.routing_extapp_intapp: eval: *terraform.EvalWriteResourceState, err: Invalid for_each argument: The “for_each” value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the for_each depends on.

2020/02/19 14:49:38 [TRACE] [walkPlan] Exiting eval tree: module.routing_extapp_intapp.aws_route.dst_stack

2020/02/19 14:49:38 [TRACE] vertex “module.routing_extapp_intapp.aws_route.dst_stack”: visit complete

2020/02/19 14:49:38 [TRACE] GRPCProvider: Close

2

Hi @vmorkunas,

Routing table ids are not good candidates to use as identifiers for instances in for_each because those ids cannot be predicted during planning: the remote system allocates them only during actual creation.

When working with for_each we must use keys built from values that are chosen in the configuration, not in the remote system. With what you’ve shared here I’m not sure exactly what values to suggest, but if your module "extapp_vpc" has some argument that somehow specifies where to create route tables – perhaps a set of availability zone names? – then I’d make that private_route_table_ids output instead be a map from whatever name the caller specified to the chosen ids.

Then you can use the keys from that map to identify your route instances, instead of the route table ids:

  for_each = {
    for rt_key, rt_id in var.routing.src_rt_ids :
    format("%s-%s", var.routing.name, rt_key) => rt_id
    if var.routing.src_provider == "aws.stack"
  }

I also have a side-note based on something else in your example: it looks like you’re building this module to allow the routes to be created between objects in different AWS provider instances aws.stack and aws.ops, and are using duplicated resource blocks to allow a switch between the two configurations based on an input variable.

The intended way to address that in Terraform is for your child module to declare its two proxy provider configurations (presumably you already have those for declaring aws.stack and aws.ops) using nomenclature that makes sense in the context of that module, such as src and dst in your case:

provider "aws" {
  alias = "src"
}

provider "aws" {
  alias = "dst"
}

Then in the calling module you can pass whichever provider configuration makes sense for each one, and remove the extra src_provider and dst_provider input variables and associated duplication in the module:

module "routing_extapp_data" {
    source = "../../modules/Stack/Routing"
    routing = { 
        src_rt_ids = module.extapp_vpc.private_route_table_ids
        dst_rt_ids = module.data_vpc.private_route_table_ids
        src_cidr = module.extapp_vpc.vpc.cidr_block
        dst_cidr = module.data_vpc.vpc.cidr_block
        peering_connection_id = module.peering_extapp_data.peering_connection_ids
        name = "extapp-data"
    }
    providers = {
        aws.src = aws.stack
        aws.dst = aws.stack
    }
}

In the above case, both aws.src and aws.dst should be pointed to aws.stack in the calling module, getting the desired effect. A different instance of this module could set them both to aws.ops instead, or set one to aws.stack and the other to aws.ops to route between them.

Your resource block for “src”'s route can then refer to aws.src:

resource "aws_route" "src" {
  provider = aws.src
  for_each = {
    for rt_key, rt_id in var.routing.src_rt_ids :
    format("%s-%s", var.routing.name, rt_key) => rt_id
  }

  # (and all the other arguments as before)
}
3

Great answer, thank you very much. Going to try this now.

4

did this work? I have a similar problem here: https://github.com/terraform-aws-modules/terraform-aws-transit-gateway/issues/2

How do I change the for_each for it to work?

5

Too late to the party but yes it works. Or better say depends on the use case. Adding a map output variable based on zone keys private_subnets_rt_ids_map lets say (instead of private_subnets_rt_ids which is a list of ids) indeed helps by itself. However, in my case I needed to collect the map from multiple iterations of a RT module (private-subnets-rt) like this:

route_table_ids_map = merge(module.private-subnets-rt.*.private_subnets_rt_ids_map)

which does not work since I hit dynamic dependancy again:

╷
│ Error: Error in function call
│ 
│   on main.tf line 82, in module "tgwa":
│   82:    route_table_ids_map = merge(module.private-subnets-rt.*.private_subnets_rt_ids_map)
│     ├────────────────
│     │ module.private-subnets-rt is a list of object, known only after apply
│ 
│ Call to function "merge" failed: arguments must be maps or objects, got "list of dynamic".

There is also the issue of avoiding getting duplicate keys (the zones) in this case (not a problem just mentioning).