For_each local cannot be determined until apply

Terraform
1

Greetings,

I have a kind of chicken-and-egg problem that I’m trying to resolve.

So I have following module:

module "kafka" {
  for_each = var.kafka  # I'd like to have multiple clusters
  source = "some_open_source"
}

By using for_each I’m trying to implement multi-cluster feature. So config would be something like:

kafka = {
  first_cluster = {
    someoption = "bbb"
    dns_prefix = "mysuperprefix"
  }
  second_cluster = {
    someoption = "ccc"
    dns_prefix = "mysuperprefix"
  }
}

Now a single Kafka cluster could provide multiple brokers (endpoints - DNS “A” records). The module return those endpoints as a list. I want to have my own custom route53 records (CNAME) pointing to whatever the module gets from AWS.

To resolve that I have following construction:

locals {
  kafka_brokers = flatten([
    for cluster_key, cluster_container in var.kafka : [
      for broker_hostname in module.kafka[cluster_key].all_brokers : {
        cluster = cluster_key
        hostname = broker_hostname
        te_hostname = "${cluster_container["dns_prefix"]}${regex("^b-([0-9]+).", broker_hostname)[0]}-aaa.${var.domain_suffix}"
      }
    ]
  ])
}

That returns something like: (did some tests)

 [
  {
    "cluster" = "first_cluster"
    "hostname" = "b-1.aaaaaaa.uzwuro.c11.kafka.us-east-1.amazonaws.com"
    "te_hostname" = "mysuperprefix1-aaa.va1.ccc.com"
  },
  {
    "cluster" = "second_cluster"
    "hostname" = "b-2.aaaaaaa.uzwuro.c11.kafka.us-east-1.amazonaws.com"
    "te_hostname" = "mysuperprefix2-aaa.va1.ccc.com"
  },

Now finally the route53

resource "aws_route53_record" "kafka" {
  for_each = local.kafka_brokers
  ...omitted...
}

The problem is I’m getting following error:

╷
│ Error: Invalid for_each argument
│
│   on kafka.tf line 78, in resource "aws_route53_record" "kafka":
│   78:   for_each = local.kafka_brokers
│     ├────────────────
│     │ local.kafka_brokers will be known only after apply
│
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.
╵

which is make sense to me

I’m trying to find a workaround or any good practice advice on how it should be done to make it work. I appreciate any help.

2

If you can’t create your for_each set using literals and data sources that don’t depend upon a module’s output then you need to use -target as suggested.

Or, you can add that aws_route53_record resource to your module. That’d be my choice.

3

I second doing the route53 work in the kafka module.

4

Thanks guys for the advice.

I guess the major point here is that I use an open-source module. I don’t want to create a fork and maintain it. So pretty much that’s why I came up with this approach that I have now, tho it’s not working :smiley:

I actually was considering using -target , just was not sure if that is considered a good practice since I’m getting “The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes” from Terraform.

I’m also wondering about some kind of intermediate module. Pretty much I’m going to create a module that includes the module.

5

You can contribute this to the module. Isn’t this what open-source is about?
It’s pretty common for modules to have an input boolean variable create_whatever that enables the creation of the whatever resource, like aws_route53_record in your case. (drop the provider name from the variable name if all the resources are created using the same provider).
Maybe other people will benefit from it as well.