Invalid for_each argument - provided tuple

caldwell2000 · August 28, 2020, 9:20pm

My goal is to use a list of Azure policy display names to populate an Azure policyset. I have most of the code working, but am having an issue flattening out my list used to look up the policy ID using the data source terraform type. I am getting the following error


$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


Error: Invalid for_each argument

  on variables.tf line 40, in data "azurerm_policy_definition" "security_policyset_definitions":
  40:   for_each     = local.security_flat

The given "for_each" argument value is unsuitable: the "for_each" argument
must be a map, or set of strings, and you have provided a value of type tuple.

Here are my variables. The issue is with calling [[[data “azurerm_policy_definition”]]], the for_each data type is not correct.


variable "policyset_definition_category" {
  type        = string
  description = "The category to use for all PolicySet defintions"
  default     = "Custom"
}

locals {
  /*  security_flat = flatten([
    for pol in var.security_policyset_definitions : {
      policy_name = pol.policy
    }
  ])
*/
  security_flat = flatten([
    for pol in var.security_policyset_definitions : {
      policy_name = pol.policy
    }
  ])

}

data "azurerm_policy_definition" "security_policyset_definitions" {
  for_each     = local.security_flat
  display_name = each.value
  ##   display_name = "Internet-facing virtual machines should be protected with network security groups"
}

variable "security_policyset_definitions" {
  type = any
  /*
  type = map(object({
    policy = string
    effect = string
  }))
*/
  description = "List of policy definitions (display names) for the security_governance policyset"
  default = [
    {
      policy = "Internet-facing virtual machines should be protected with network security groups"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Subnets should be associated with a Network Security Group"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Gateway subnets should not be configured with a network security group"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Storage accounts should restrict network access"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Secure transfer to storage accounts should be enabled"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Access through Internet facing endpoint should be restricted"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Storage accounts should allow access from trusted Microsoft services"
      effect = "AuditIfNotExists"
    },
    {
      policy = "RDP access from the Internet should be blocked"
      effect = "AuditIfNotExists"
    },
    {
      policy = "SSH access from the Internet should be blocked"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Disk encryption should be applied on virtual machines"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Automation account variables should be encrypted"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Azure subscriptions should have a log profile for Activity Log"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Email notification to subscription owner for high severity alerts should be enabled"
      effect = "AuditIfNotExists"
    },
    {
      policy = "A security contact email address should be provided for your subscription"
      effect = "AuditIfNotExists"
    },
    {
      policy = "Enable Azure Security Center on your subscription"
      effect = "AuditIfNotExists"
    }
  ]
}

variable "iam_policyset_definitions" {
  type        = list
  description = "List of policy definitions (display names) for the iam_governance policyset"
  default = [
    "Audit usage of custom RBAC rules",
    "Custom subscription owner roles should not exist",
    "Deprecated accounts should be removed from your subscription",
    "Deprecated accounts with owner permissions should be removed from your subscription",
    "External accounts with write permissions should be removed from your subscription",
    "External accounts with read permissions should be removed from your subscription",
    "External accounts with owner permissions should be removed from your subscription",
    "MFA should be enabled accounts with write permissions on your subscription",
    "MFA should be enabled on accounts with owner permissions on your subscription",
    "MFA should be enabled on accounts with read permissions on your subscription",
    "There should be more than one owner assigned to your subscription"
  ]
}

variable "data_protection_policyset_definitions" {
  type        = list
  description = "List of policy definitions (display names) for the data_protection_governance policyset"
  default = [
    "Azure Backup should be enabled for Virtual Machines",
    "Long-term geo-redundant backup should be enabled for Azure SQL Databases",
    "Audit virtual machines without disaster recovery configured",
    "Key Vault objects should be recoverable"
  ]
}

Here is the main.tf:


terraform {
  required_version = ">= 0.12.21"
}

provider "azurerm" {
  version = "~> 2.25.0"
  features {}
}

resource "azurerm_policy_set_definition" "sec_policy_set" {
  name         = "a"
  policy_type  = "Custom"
  display_name = "c"
  description  = "d"

  metadata = < y }

    content {
      #policy_definition_id = policy_definition_reference.value.policy
      #policy_definition_id = policy_definition_reference.value["policy"]
      policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
      parameters = {
        Effect = policy_definition_reference.value["effect"]
      }
      /*
      policy_definition_id = policy_definition_reference.value["policy_description"]
      parameters = {
        Effect = policy_definition_reference.value["effect"]
      }
      */
    }
  }
}

I have hard-coded a valid policy_definition_id in the azurerm_policy_set_definition resource and if I change the “display_name” property of the data lookup function “azurerm_policy_definition” for now to validate that that resource works properly. Showing the change below:


data "azurerm_policy_definition" "security_policyset_definitions" {
  #for_each     = local.security_flat
  #display_name = each.value
  display_name = "Internet-facing virtual machines should be protected with network security groups"
}

After making these minor changes, my plan executes, but is using the static value for “policy_definition_id”. I need to be able to look up the policy_definition_id, by using the data source type:

Ref: https://www.terraform.io/docs/providers/azurerm/d/policy_set_definition.html


$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.azurerm_policy_definition.security_policyset_definitions: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_policy_set_definition.sec_policy_set will be created
  + resource "azurerm_policy_set_definition" "sec_policy_set" {
      + description           = "d"
      + display_name          = "c"
      + id                    = (known after apply)
      + management_group_id   = (known after apply)
      + management_group_name = (known after apply)
      + metadata              = jsonencode(
            {
              + category = "Custom"
            }
        )
      + name                  = "a"
      + policy_definitions    = (known after apply)
      + policy_type           = "Custom"

      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
      + policy_definition_reference {
          + parameters           = {
              + "Effect" = "AuditIfNotExists"
            }
          + policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/b821191b-3a12-44bc-9c38-212138a29ff3"
          + reference_id         = (known after apply)
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Thank You for any help you can provide to get me over this hump!

apparentlymart · August 31, 2020, 4:31pm

Hi @caldwell2000,

It seems like your goal is to have one data.azurerm_policy_definition.security_policyset_definitions instance per element of local.security_flat. When you use for_each Terraform needs to know which string to use to uniquely identify each of the instances it will produce, which it does by taking either the keys of the map you provide or the elements of the set of strings you provide.

The upshot of this is that you’ll need to define some sort of unique identifier for each element, which Terraform will then use as the identifier for the corresponding instance. Looking at your declaration of variable "security_policyset_definitions", it seems like there is no suitable identifier there right now: you only have a human-friendly English description of the policy and a non-unique “effect”.

One way to proceed here would be to change the type constraint of variable "security_policyset_definitions" to be a map of objects instead, where the map keys will be the unique identifiers for your definitions. Interestingly, I see that you already have a suitable type argument commented out in what you shared:

  type = map(object({
    policy = string
    effect = string
  }))

If you declare it in that way (and make sure the default complies with that new type) you can then use this variable directly as the for_each. (It doesn’t seem like you actually need flatten here, because the data structure is already a flat map of objects):

data "azurerm_policy_definition" "security_policyset_definitions" {
  for_each = var.security_policyset_definitions

  display_name = each.value.policy
}

I’m not familiar with this azurerm_policy_definition data source, but it seems like it supports looking up policies both by display name (non-unique) and name (unique). With that said, one design decision you could potentially make here, if the unique names are predictable, is to make the map keys be the unique names in the remote system and use name = each.key instead of display_name = each.value.policy, though I don’t know if that actually makes sense for what you are trying to achieve here.

caldwell2000 · September 22, 2020, 2:17pm

Thank you. That was very helpful and problem solved.

Topic		Replies	Views
"Invalid for_each argument" with resource attributes Terraform	2	854	October 12, 2021
“Invalid for_each argument” with flattened maps Terraform	2	584	May 21, 2021
Invalid for_each argument Terraform	8	8033	January 26, 2022
For_each invalid argument - values derived from resources attributes Terraform	3	1471	November 9, 2022
"Invalid for_each argument" during import Terraform	7	1081	June 22, 2023

Invalid for_each argument - provided tuple

Related topics