Issue using map(object) variable

Terraform
1

I am trying to create Databricks Workspaces using a module with for_each loop that references a map(object). I am also creating a Private Endpoint in Azure using a separate module for each workspace.

However the issue I am having is in the module for PE we use a dynamic block to create the IP configuration. Which references a map(object) variable with the configuration such as name, private ip address, sub resource name and member name.

In the map(object) I have created three keys with the values for each databricks workspace.

So when calling the Private Endpoint module, I set ipconfigs = var.databricks_pvtep_ip_configs

What I want to happen is the module to loop through each and only use the ip configuration for each workspace. I have tried setting for_each workspaces and each.key but when terraform plan runs it applies all of the ip configuration in the map(object) to all three workspaces. Not one for each.

So really struggling to get this work. A workaround is to call the module three times and set the ip configs variable individually but want to avoid this and call the module once.

Please see the code below

PE Module:

resource "azurerm_private_endpoint" "pvt_ep" {
  name                = "pvtep-${var.pvt_ep_name}"
  location            = var.location
  resource_group_name = var.rg_name
  subnet_id           = var.subnet_id
  custom_network_interface_name = try(var.custom_network_interface_name, "nic-pvtep-${var.pvt_ep_name}")

  dynamic "private_dns_zone_group" {
    for_each = var.private_dns_zone_group_setup == true ? [1] : []
    content {  
      name              = var.pvt_dnszone_name
      private_dns_zone_ids = var.pvt_dnszone_ids
    }
  }

  private_service_connection {
    name                              = var.pvt_svc_conn_name
    private_connection_resource_id    = var.pvt_conn_res_id
    private_connection_resource_alias = var.private_connection_resource_alias
    is_manual_connection              = var.is_manual_connection
    subresource_names                 = try(var.subresource_ids, null)
    request_message                = (var.is_manual_connection == true) ? var.request_message : null
  }

  dynamic "ip_configuration" {
    for_each = var.ip_configs
    content {
      name = ip_configuration.value.ip_config_name # - (Required) Specifies the Name of the IP Configuration. Changing this forces a new resource to be created.
      private_ip_address = ip_configuration.value.ip_config_private_ip_address #- (Required) Specifies the static IP address within the private endpoint's subnet to be used. Changing this forces a new resource to be created.
      subresource_name = ip_configuration.value.ip_config_subresource_name # (Optional) Specifies the subresource this IP address applies to. subresource_names corresponds to group_id. Changing this forces a new resource to be created.
      member_name = ip_configuration.value.ip_config_member_name #
    }  
  } 
  tags = var.tags_common
}

Module Call:

module "lgr-databricks-workspaces" {
  for_each                                             = var.databricks_workspaces
  source                                               = "../../LGR-DataPlatform-Databricks/02.LandingZone/02.AzureDatabricks/adb_v1"
  databricks_workspace_name                            = each.value.workspace_name
  databricks_rg_name                                   = var.databricks_rg_name
  location                                             = var.location
  databricks_sku                                       = var.databricks_sku
  infrastructure_encryption_enabled                    = var.infrastructure_encryption_enabled
  public_network_access_enabled                        = var.public_network_access_enabled
  network_security_group_rules_required                = var.network_security_group_rules_required
  managed_rg_name                                      = each.value.managed_rg_workspace_name          
  no_public_ip                                         = var.no_public_ip
  virtual_network_id                                   = var.virtual_network_id
  databricks_public_subnet_name                        = each.value.databricks_public_subnet_name
  databricks_private_subnet_name                       = each.value.databricks_private_subnet_name
  public_subnet_network_security_group_association_id  = each.value.public_subnet_network_security_group_association_id
  private_subnet_network_security_group_association_id = each.value.private_subnet_network_security_group_association_id
  tags_common                                          = local.tags_databricks_workspace
}

module "lgr-databricks-workspace-pe" {
  source                     = "./TerraModules/37.PrivateEPModule"
  for_each                   = var.databricks_workspaces
  location                   = var.location
  rg_name                    = var.databricks_rg_name
  subnet_id                  = var.pvtep_subnet_id
  pvt_ep_name                = each.value.databricks_private_pvt_ep_name
  pvt_dnszone_name           = local.backend_pvt_dnszone_name
  pvt_dnszone_ids            = local.backend_pvt_dnszone_ids
  pvt_svc_conn_name          = each.value.databricks_private_pvt_ep_name
  pvt_conn_res_id            = module.lgr-databricks-workspaces[each.key].workspace_id
  subresource_ids            = local.backend_pvt_dnszone_ids
  tags_common                = local.tags_databricks_backend
  ip_configs                 = var.databricks_pvtep_ip_configs[each.key]

}

Variable:

variable "databricks_pvtep_ip_configs" {
  type   = map(object({
    ip_config_name                   = string
    ip_config_private_ip_address     = string
    ip_config_subresource_name       = string
    ip_config_member_name            = string 
  }))
  default = {
   lgr-workspace1 = {
      ip_config_name                   = "lgr-workspace1-config"
      ip_config_private_ip_address     = "{{databricks_workspace1_pvtep_ip_address}}"
      ip_config_subresource_name       = "databricks_ui_api"
      ip_config_member_name            = "databricks_ui_api"
    }
   lgr-workspace2 = {
      ip_config_name                   = "lgr-workspace2-config"
      ip_config_private_ip_address     = "{{databricks_workspace2_pvtep_ip_address}}"
      ip_config_subresource_name       = "databricks_ui_api"
      ip_config_member_name            = "databricks_ui_api"
    }
   lgr-workspace3 = {
      ip_config_name                   = "lgr-workspace3-config"
      ip_config_private_ip_address     = "{{databricks_workspace3_pvtep_ip_address}}"
      ip_config_subresource_name       = "databricks_ui_api"
      ip_config_member_name            = "databricks_ui_api"
    }
  }
}
2

In the screen shots you can see the three IP configs being created. The working one when one is created is when I use three module calls.

3

Please fix the mangled formatting of your code: Welcome to the forum - please reformat your message

4

Hi Max, thanks hopefully that is better now added ```

5

You haven’t posted any.

This description of behaviour doesn’t seem to match what the code would actually do.

Have you been experimenting with different versions, and not pasted the same version of the code as what you are describing?

In any case, some actual Terraform plan or error message output would be very useful to help understand your actual problem.

It is much preferred if you do not post it as screenshots - copy/pasted text (suitably enclosed in ```) is much easier to read - and quote.

6

The error I am experiencing when running terraform plan with

ip_configs = var.databricks_pvtep_ip_ipconfigs[each.key]

is the following:

 Error: Invalid value for input variable
│ 
│   on xxx-test-multi-workspace-module.tf line 35, in module "lgr-databricks-workspace-pe":
│   35:   ip_configs                 = var.databricks_pvtep_ip_configs[each.key]
│ 
│ The given value is not suitable for
│ module.lgr-databricks-workspace-pe.var.ip_configs declared at
│ TerraModules/37.PrivateEPModule/01.Input.tf:43,1-22: element
│ "ip_config_private_ip_address": object required.
╵

validation fails

I want to apply the following configuration to each workspace:

variable "databricks_pvtep_ip_configs" {
 type   = map(object({
   ip_config_name                   = string
   ip_config_private_ip_address     = string
   ip_config_subresource_name       = string
   ip_config_member_name            = string 
 }))
 default = {
  lgr-workspace1 = {
     ip_config_name                   = "lgr-workspace1-config"
     ip_config_private_ip_address     = "{{databricks_workspace1_pvtep_ip_address}}"
     ip_config_subresource_name       = "databricks_ui_api"
     ip_config_member_name            = "databricks_ui_api"
   }
  lgr-workspace2 = {
     ip_config_name                   = "lgr-workspace2-config"
     ip_config_private_ip_address     = "{{databricks_workspace2_pvtep_ip_address}}"
     ip_config_subresource_name       = "databricks_ui_api"
     ip_config_member_name            = "databricks_ui_api"
   }
  lgr-workspace3 = {
     ip_config_name                   = "lgr-workspace3-config"
     ip_config_private_ip_address     = "{{databricks_workspace3_pvtep_ip_address}}"
     ip_config_subresource_name       = "databricks_ui_api"
     ip_config_member_name            = "databricks_ui_api"
   }
 }
}

But if I just use

ip_configs                 = var.databricks_pvtep_ip_configs

It applies all three to all three

Not one per workspace

+ ip_configuration {
          + member_name        = "databricks_ui_api"
          + name               = "lgr-workspace1-config"
          + private_ip_address = "10.141.120.71"
          + subresource_name   = "databricks_ui_api"
        }
      + ip_configuration {
          + member_name        = "databricks_ui_api"
          + name               = "lgr-workspace2-config"
          + private_ip_address = "10.141.120.72"
          + subresource_name   = "databricks_ui_api"
        }
      + ip_configuration {
          + member_name        = "databricks_ui_api"
          + name               = "lgr-workspace3-config"
          + private_ip_address = "10.141.120.73"
          + subresource_name   = "databricks_ui_api"
        }

Rather than…

+ ip_configuration {
          + member_name        = "databricks_ui_api"
          + name               = "lgr-workspace1-config"
          + private_ip_address = "10.141.120.71"
          + subresource_name   = "databricks_ui_api"
        }

then second then third when creating the private endpoints

7

You have already figured out part of the correct solution yourself:

You then encountered a validation error, which you should resolve by updating the validation to be consistent with the change you had just made.

You also should stop using dynamic for the ip_configuration block, since you don’t intend for the number of ip_configuration blocks to be dynamic!

8

ok so how should I update the validation then? As not sure how to do that and what you mean exactly?

I did wonder whether the dynamic block with for_each loop was part of the problem. This was a module I was consuming from another location and not written by me personally.

Any guidance you can provide would be appreciated.
thanks

9

You haven’t disclosed the contents of the module’s variable "ip_configs" block which makes it a bit tricky to advise further.

Are you open to changing the module, or is it something that is used in other places, that needs to have its interface stay constant?

10

The modules variable is the following:

variable "ip_configs" {
  type   = map(object({
    ip_config_name                   = string
    ip_config_private_ip_address     = string
    ip_config_subresource_name       = string
    ip_config_member_name            = string 
  }))
  default = {}
}

I can change the module if it is necessary, it is used in other places but I can make a copy of it and use it locally in my code.

Let me know what is best to get this to work.

11

Ah, well, we might as well avoid forking the module then - no need to create extra work.

The definition of the ip_configs variable is a bit weird. It specifies it’s a map, but, at least in the bits you shown, doesn’t appear to actually use the map key for anything at all.

So I guess I would try:

ip_configs = { this_map_key_is_ignored_so_it_does_not_matter_what_goes_here = var.databricks_pvtep_ip_ipconfigs[each.key] }
1 Like
12

ok thanks, will give it a try. Could you explain this line a bit more, what would I put in it’s place?

this_map_key_is_ignored_so_it_does_not_matter_what_goes_here

13

You can just write it exactly like that. I was just making it abundantly clear that there needed to be a string there, but (as far as I can see) the string is never used.

1 Like
14

Hi Max, that is wonderful thank you. I just ran it and it works a treat!! I have been struggling for days on this!!

  ip_configs                 = { this_map_key_is_ignored_so_it_does_not_matter_what_goes_here = var.databricks_pvtep_ip_configs[each.key] }

I corrected the name of the variable slightly to match.

now applying the map of objects correctly

 + ip_configuration {
          + member_name        = "databricks_ui_api"
          + name               = "lgr-workspace3-config"
          + private_ip_address = "10.141.120.73"
          + subresource_name   = "databricks_ui_api"
        }

Clever how that works so needed to provide an empty string first.