For_each with list of unknown strings

Terraform
1

I’m trying to setup a simple demo where I create an azurerm_linux_web_app resource, grab it’s possible_outbound_ip_address_list property and use that to create firewall rules using the azurerm_mariadb_firewall_rule resource.

This is what I wrote for that resource:

resource "azurerm_mariadb_firewall_rule" "allow_web_to_db" {
  for_each = toset(azurerm_linux_web_app.web.possible_outbound_ip_address_list)

  name                = "web_app_${replace(each.value, ".", "-")}"
  server_name         = azurerm_mariadb_server.db.name
  resource_group_name = azurerm_mariadb_server.db.resource_group_name
  start_ip_address    = each.value
  end_ip_address      = each.value
}

Unfortunately it seems Terraform refuses to apply this until the resource already exists. It appears it wants to know what keys it’s going to use before applying.

╷
│ Error: Invalid for_each argument
│
│   on main.tf line 87, in resource "azurerm_mariadb_firewall_rule" "allow_web_to_db":
│   87:   for_each = azurerm_linux_web_app.web.possible_outbound_ip_address_list
│     ├────────────────
│     │ azurerm_linux_web_app.web.possible_outbound_ip_address_list is a list of string, known only after apply
│
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this
│ resource.
│
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map values.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.
╵

Why can’t terraform just skip this and say “known after apply”?

Surely it can’t be this difficult to just deploy a resource and whitelist some IP’s without requiring 2 deploy stages.

I’ve been searching around for solutions / workarounds for this, but have been unable to succesfully do so. Hopefully someone here can provide a solution to this.

2

Hi @xorinzor,

The problem in this case is that Terraform doesn’t even have enough information to know the addresses of the resource instances being planned. It’s valid to have unknown values in the configuration arguments for a particular resource instance, but Terraform always needs to know which resource instances ought to exist because otherwise it wouldn’t be able to make any predictions at all about what might come to exist – this could produce zero instances, four instances, or thousands of instances.

Terraform needs to assign each resource instance a unique address so it can track the objects from the plan phase into the apply phase, and thus make sure the applied changes are consistent with what was planned. A resource instance address for a resource with for_each includes the keys from the given map, giving addresses like this:

  • azurerm_mariadb_firewall_rule.allow_web_to_db["a"]
  • azurerm_mariadb_firewall_rule.allow_web_to_db["b"]
  • azurerm_mariadb_firewall_rule.allow_web_to_db["c"]

(I’m just using single letters as placeholders because I’m not familiar with azurerm_linux_web_app so I don’t know what real values of possible_outbound_ip_address_list might look like.)

Without knowing the "a", "b", and "c" portions of those addresses, these resource instances would not have fully-qualified unique keys, and so Terraform would have no way to track these objects from the plan phase into the apply phase. This is just one of the design compromises included in Terraform to help ensure that the apply phase will always take the actions proposed in the plan or return an error explaining why it cannot.

For situations like yours where the problem only arises during an initial “create from nothing” operation, a typical approach is to use a two-step process only for the initial creation:

  1. terraform apply -target='azurerm_linux_web_app.web'
  2. terraform apply

Then once you’ve completed that initial bootstrapping process you can just use normal terraform apply with no arguments as long as you never need to replace the azurerm_linux_web_app.web object.

There is an existing GitHub issue representing a goal of making this possible without encountering any errors:

However, in all of my research so far the most compelling option has seemed to be for Terraform to do something similar to automatically detecting the problem and just excluding azurerm_mariadb_firewall_rule.allow_web_to_db from the plan automatically itself, without you needing to use -target manually to do it.

That does seem to be the most likely future compromise, but with some remaining design questions about whether Terraform can and should still try to produce at least a partial plan for the objects it has “deferred” to a future run, so that it’s less likely that someone will apply a first round of changes and then encounter a difficult-to-resolve error or configuration problem on the second round. I am hoping to continue researching this some more later in the year once I have a break from some other work, and so hopefully we’ll be able to make this work a little more gracefully – though probably still requiring two steps – in a future Terraform release.

3

Thank you for the detailed explanation, I read a few posts online but didn’t really understand them, this one explains it much better!

Am I correct in deducting from this that it mostly boils down to Terraform being unable to determine if & what unique keys to use for the resources to-be-created?

IMO, the amount of resources it creates, whether one or a thousand, shouldn’t be something for TF to be concerned with, as that should be the responsibility of the developer. As long as the code is correct, and the logic of TF is correct, the outcome should be exactly as defined by the developer.

I’m very interested to see if, and what, solution you can potentially come up with; I’ll definitely be following the mentioned GH issue.

The behaviour I’d personally expect from TF is:

  • during planning phase indicate that TF can’t determine the resources until apply yet (optionally require a CLI option to be passed to continue regardless, this would indicate that the user understands & accepts the risk)
  • In the apply stage, after azurerm_linux_web_app.web is created, the possible_outbound_ip_address_list property is known and TF will do it’s check for key uniqueness. If there’s an issue, error out (after all, TF couldn’t know this at planning stage, the developer knows this and takes that risk by proceeding)
  • Assuming no errors, continue the apply as normal

I’ll use the -target solution for now in my demo and give them the context from your explanation, thank you!