Problems using for_each using local variable created based on another local

Terraform
,
1

Hi all,

I am trying to create azure keyvault secrets using locals which reference data resources. I am iterating over an array containing my environments and creating a list of maps where
each item is the set of secrets for a given environment.
Using another local, I then proceed to merge these maps into a single one by creating two lists, one with keys and another with values and then zipping them.

I finally use for_each on the second local to create the resource.

If I run my root module without creating the actual secret resources ("azurerm_key_vault_secret) and a second time with it, it all works fine.

If I try to do it all in one go, as I want to implement on my CI/CD I get the error message:

│ Error: Invalid for_each argument
│
│ on variables.tf line 239, in resource “azurerm_key_vault_secret” “example”:
│ 239: for_each = nonsensitive(local.example_map)
│ ├────────────────
│ │ local.example_map will be known only after apply
│
│ The “for_each” value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target
│ argument to first apply only the resources that the for_each depends on.

Could you please help me? I have read many of questions related to for_each in this forum but haven’t found one quite the same usecase.
Maybe I am going about the whole thing wrong. Any pointers would be highly appreciated.

Here is the code I am trying to make work:

variable "environment" {
    default = [ "dev", "prod"]
}

locals {
  example = distinct(flatten([
    for namespace in var.environment : {
        "${environment}-password1" = "${environment}-password",
        "${environment}-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["${environment}"].primary_connection_string}\"}",
        "${environment}-password3" = "{\"client_id\" : \"${jsondecode("${data.azurerm_key_vault_secret.other_credentials["${environment}"].value}").clients["example"].client_id}\"}",
        "${environment}-password4" = "{\"password\" : \"${data.azurerm_key_vault_secret.k_password.value}\"}",
        "${environment}-password5" = "{\"azurestorageaccountname\" : \"${data.azurerm_storage_account.example.name}\", \"azurestorageaccountkey\" : \"${data.azurerm_storage_account.example.primary_access_key}\"}",
        "${environment}-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}",
  }]))

  example_map = zipmap(
    flatten(
      [for item in local.example : keys(item)]
    ),
    flatten(
      [for item in local.example : values(item)]
    )
  )
}

resource "azurerm_key_vault_secret" "example" {
  for_each     = nonsensitive(local.example_map)
  name         = each.key
  value        = each.value
  key_vault_id = module.keyvault.id
  content_type = "password"
}

Here is the data structures created by local.example and local.example_map


"example": {
  "value": [
    {
      "dev-password1" = "dev-password",
      "dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "dev-password3" = "{\"client_id\" : \"myclientID\"}",
      "dev-password4" = "{\"password\" : \"password123\"}",
      "dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "dev-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
    },
    {
      "prod-password1" = "prod-password",
      "prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "prod-password3" = "{\"client_id\" : \"myclientID\"}",
      "prod-password4" = "{\"password\" : \"password123\"}",
      "prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
    }
  ]
}

"example_map": {
    "value": {
      "dev-password1" = "dev-password",
      "dev-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "dev-password3" = "{\"client_id\" : \"myclientID\"}",
      "dev-password4" = "{\"password\" : \"password123\"}",
      "dev-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "dev-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
      "prod-password1" = "prod-password",
      "prod-password2" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}",
      "prod-password3" = "{\"client_id\" : \"myclientID\"}",
      "prod-password4" = "{\"password\" : \"password123\"}",
      "prod-password5" = "{\"azurestorageaccountname\" : \"somestorageaccount\", \"azurestorageaccountkey\" : \"XXXxxxxXXXXxxxxXXXxxxxxxe++++++NNNNNNNNNCCCccccccccccccccccc==}\"}",
      "prod-password6" = "{\"connection_string\" : \"DefaultEndpointsProtocol=https;AccountName=yetanotherone;AccountKey=blablablablblabalbalbalbalblablablablablalbalbalbl==;EndpointSuffix=foo.bar.net\"}"
    },
    "type": [
    "object",
    {
        "dev-password1": "string",
        "dev-password2": "string",
        "dev-password3": "string",
        "dev-password4": "string",
        "dev-password5": "string",
        "dev-password6": "string",
        "prod-password1": "string",
        "prod-password2": "string",
        "prod-password3": "string",
        "prod-password4": "string",
        "prod-password5": "string",
        "prod-password6": "string",
    }
    ]
}
2

A little update from my side on this. I did find out that removing the reference to another module will make it work. I thought then the problem is that for_each couldn’t handle data coming from other modules.

"${environment}-password6" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"

However, and this is what confuses me the most is that if I work with the following data structure (see below), which is hardcoding the data I am passing to the for_each instead of doing the first transformation based on namespaces. It works fine and I can use for_each together with data coming from modules.

locals {
  hardcoding_namespaces = {
    "dev-password1" = "dev-password"
    "dev-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["dev"].primary_connection_string}\"}"
    "dev-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"
    "prod-password1" = "prod-password"
    "prod-password2" = "{\"connection_string\" : \"${data.azurerm_storage_account.storage_account_example["prod"].primary_connection_string}\"}"
    "prod-password3" = "{\"connection_string\" : \"${module.some_module.connection_string}\"}"

  }
}

resource "azurerm_key_vault_secret" "another_example" {
  for_each     = local.hardcoding_namespaces
  name         = each.key
  value        = each.value
  key_vault_id = module.keyvault.id
  content_type = "password"

}

Anyways, I will resolve my problem removing this data for now but I still not quite sure why Terraform handles well when this data is passed directly but won’t work if I am doing some transformation within the locals block. If anybody has an idea, please :slight_smile: