Is there a way to use a function to run a loop on an input object based a range

I originally posted this on Stack Overflow because I was not aware of this forum.

I have created a module with the following

resource "aci_ranges" "add_vlan" {
  for_each      = local.vlan_list
  alloc_mode    = each.value["alloc_mode"]
  annotation    = each.value["annotation"]
  name_alias    = each.value["name_alias"]
  vlan_pool_dn  = each.value["vlan_pool"]
  role          = each.value["role"]
  from          = "vlan-${each.value["from"]}"
  to            = "vlan-${each.value["to"]}"
}

From Here I have defined a variables file to make it so users don’t have to enter every variable… they can accept defaults

terraform {
  experiments = [module_variable_optional_attrs]
}

variable "vlan_list" {
  description = "Add VLANs to VLAN Pools"
  type = map(object({
    alloc_mode  = optional(string)
    annotation  = optional(string)
    from        = optional(number)
    name_alias  = optional(string)
    role        = optional(string)
    to          = optional(number)
    vlan_pool   = optional(string)
  }))
}

locals {
  vlan_list = {
    for k, v in var.vlan_list : k => {
      alloc_mode  = coalesce(v.alloc_mode, "static")
      annotation  = (v.annotation != null ? v.annotation : "")
      from        = (v.from != null ? v.from : 1)
      name_alias  = (v.name_alias != null ? v.name_alias : "")
      role        = coalesce(v.role, "external")
      to          = coalesce(v.to, 1)
      vlan_pool   = (v.vlan_pool != null ? v.vlan_pool : "")
    }
  }
}

Here is what a user would enter then to consume the module (note that I know the for statements are not correct at all but are just a reference to what I would like to accomplish):

module "vlan_list" {
  depends_on  = [module.vlan_pools]
  source      = "../modules/add_vlans"
   vlan_list = {
      for i in range(1, 100):
        "access" => {
          vlan_pool   = module.vlan_pools.vlan_pool["access"]
          from        = i
          to          = i
        }...
      for i in ranges([1000-1200], [1300-1400]):
        "vmm_dynamic" => {
          alloc_mode  = "dynamic"
          vlan_pool   = module.vlan_pools.vlan_pool["vmm_dynamic"]
          from        = i
          to          = i
        }...
      for i in list[4, 100, 101]:
        "l3out" => {
          vlan_pool   = module.vlan_pools.vlan_pool["l3out"]
          from        = i
          to          = i
        }...
   }
}

When the resource creates the entries, from the API, if I do it in a range as shown below it can be problematic. In example; if someone needed to change the range for the first pool (in example) to 1-50,52-99, it would delete the entire VLAN range and then re-create it. whereas if they are creating the entry with each entry being created individually via a range loop then they can delete individual entries without affecting the entire pool.

I can do the following and it works fine… but as mentioned above; being able to add the VLANs individually from a loop would be preferable.

module "vlan_list" {
  depends_on  = [module.vlan_pools]
  source      = "../modules/add_vlans"
  vlan_list = {
    "access" = {
      vlan_pool   = module.vlan_pools.vlan_pool["access"]
      from        = 1
      to          = 99
    },
    "vmm_dynamic" = {
      alloc_mode  = "dynamic"
      vlan_pool   = module.vlan_pools.vlan_pool["vmm_dynamic"]
      from        = 1000
      to          = 1199
    },
    "l3out_1" = {
      vlan_pool   = module.vlan_pools.vlan_pool["l3out"]
      from        = 4
      to          = 4
    },
    "l3out_2" = {
      vlan_pool   = module.vlan_pools.vlan_pool["l3out"]
      from        = 100
      to          = 101
    },
  }
}

Thanks in advance for help on this.

Hi @scottyso!

With more complex object structures, I find it can be tricky to balance the iteration behavior of a collection functions with the provider. I try to avoid the passing of maps of object types into modules. As a module user, I’d be more inclined to use a module for_each to attach a list of VLANs to each pool.

Needs to be compiled, not validated. Something like this:

module "vlan_list" {
  for_each        = module.vlan_pools.vlan_pool
  depends_on      = [module.vlan_pools]
  source          = "../modules/add_vlans"
  vlan_pool       = each.key
  vlan_list       = local.vlan_list[each.key]
}

If you do need to keep the complex input, I think you can achieve the reference with the merge function.

module "vlan_list" {
  depends_on  = [module.vlan_pools]
  source      = "../modules/add_vlans"
  vlan_list   = merge(
    {
      for i in range(1, 100) :
      "access" => {
        vlan_pool = "module.vlan_pools.vlan_pool["access"]"
        from      = i
        to        = i
      }...
    },
    {
      for i in concat(range(1000, 1200), range(1300, 1400)) :
      "vmm_dynamic" => {
        alloc_mode = "dynamic"
        vlan_pool  = module.vlan_pools.vlan_pool["vmm_dynamic"]
        from = i
        to   = i
      }...
    },
    {
      for i in list(4, 100, 101) :
      "l3out" => {
        vlan_pool = module.vlan_pools.vlan_pool["l3out"]
        from = i
        to   = i
      }...
    }
  )
}

Looking forward to hearing what works out!

Thank you joatmon08… I will start testing this and see if it works. What you have shown makes sense and it seems to be what I have been missing… Let me see if I can get it to work.

So I believe I need to keep the complex input structure because it is with object type that I am able to have the different input types of number, string, Boolean etc… this is just one example where I am only using number and string.

I tried what was shared in your second example and It gives me the feedback that I need an object type for the variable. I am sorry I don’t really understand how it is an object without the function and then not an object type with the function… that is beyond what I understand.

Here is what I defined:

module "vlan_list" {
  depends_on  = [module.vlan_pools]
  source      = "../modules/add_vlans"
  vlan_list = merge(
    {
      for i in range(1, 100) :
      "access" => {
        vlan_pool   = module.vlan_pools.vlan_pool["access"]
        from        = i
        to          = i
      }...
    },
    {
      for i in concat(range(1000, 1200), range(1300, 1400)) :
      "vmm_dynamic" => {
        alloc_mode  = "dynamic"
        vlan_pool   = module.vlan_pools.vlan_pool["vmm_dynamic"]
        from        = i
        to          = i
      }...
    },
    {
      for i in list(4, 100, 101) :
      "l3out" => {
        vlan_pool   = module.vlan_pools.vlan_pool["l3out"]
        from        = i
        to          = i
      }...
    }
  )
}

Here is the error message I receive when I try to run the plan

terraform plan -out=main.plan

Warning: Experimental feature "module_variable_optional_attrs" is active

  on ../modules/add_vlans/variables.tf line 2, in terraform:
   2:   experiments = [module_variable_optional_attrs]

Experimental features are subject to breaking changes in future minor or patch
releases, based on feedback.

If you have feedback on the design of this feature, please open a GitHub issue
to discuss it.

(and 7 more similar warnings elsewhere)


Error: Invalid value for module argument

  on pools_vlan.tf line 30, in module "vlan_list":
  30:   vlan_list = merge(
  31:     {
  32:       for i in range(1, 100) :
  33:       "access" => {
  34:         vlan_pool   = module.vlan_pools.vlan_pool["access"]
  35:         from        = i
  36:         to          = i
  37:       }...
  38:     },
  39:     {
  40:       for i in concat(range(1000, 1200), range(1300, 1400)) :
  41:       "vmm_dynamic" => {
  42:         alloc_mode  = "dynamic"
  43:         vlan_pool   = module.vlan_pools.vlan_pool["vmm_dynamic"]
  44:         from        = i
  45:         to          = i
  46:       }...
  47:     },
  48:     {
  49:       for i in list(4, 100, 101) :
  50:       "l3out" => {
  51:         vlan_pool   = module.vlan_pools.vlan_pool["l3out"]
  52:         from        = i
  53:         to          = i
  54:       }...
  55:     }
  56:   )

The given value is not suitable for child module variable "vlan_list" defined
at ../modules/add_vlans/variables.tf:5,1-21: element "access": object
required.

I took a look at the output and it seems that it is a map(list(object())). I think if you change the vlan_list variable type to:

variable "vlan_list" {
  description = "Add VLANs to VLAN Pools"
  type = map(list(object({
    alloc_mode  = optional(string)
    annotation  = optional(string)
    from        = optional(number)
    name_alias  = optional(string)
    role        = optional(string)
    to          = optional(number)
    vlan_pool   = optional(string)
  })))
}

It may change some iteration you do in the child module, hopefully it is the correct type structure.

Just to close the loop on this; Below is what I have finally come up with as a solution yesterday.

variable "vlan_pools" {
  default = {
    "default" = {
      alias           = ""
      allocation_mode = "dynamic"
      description     = ""
      encap_blocks = {
        "default" = {
          allocation_mode = "inherit"
          description     = ""
          role            = "external"
          vlan_range      = "**REQUIRED**"
        }
      }
    }
  }
  type = map(object(
    {
      alias           = optional(string)
      allocation_mode = optional(string)
      description     = optional(string)
      encap_blocks = map(object(
        {
          allocation_mode = optional(string)
          description     = optional(string)
          role            = optional(string)
          vlan_range      = string
        }
      ))
    }
  ))
}

Which I then manipulate the map of objects with locals by looping through the data 4 times.

  # This first loop is to handle optional attributes and return 
  # default values if the user doesn't enter a value.
  vlan_pools = {
    for k, v in var.vlan_pools : k => {
      alias           = v.alias != null ? v.alias : ""
      allocation_mode = v.allocation_mode != null ? v.allocation_mode : "dynamic"
      description     = v.description != null ? v.description : ""
      encap_blocks    = v.encap_blocks != null ? v.encap_blocks : {}
    }
  }

  /*
  Loop 1 is to determine if the vlan_range is:
  * A Single number 1
  * A Range of numbers 1-5
  * A List of numbers 1-5,10-15
  And then to return these values as a list
  */
  vlan_ranges_loop_1 = flatten([
    for key, value in local.vlan_pools : [
      for k, v in value.encap_blocks : {
        allocation_mode = v.allocation_mode != null ? v.allocation_mode : "inherit"
        description     = v.description != null ? v.description : ""
        key1            = key
        key2            = k
        role            = v.role != null ? v.role : "external"
        vlan_split = length(regexall("-", v.vlan_range)) > 0 ? tolist(split(",", v.vlan_range)) : length(
          regexall(",", v.vlan_range)) > 0 ? tolist(split(",", v.vlan_range)
        ) : [v.vlan_range]
        vlan_range = v.vlan_range
      }
    ]
  ])

  # Loop 2 takes a list that contains a "-" or a "," and expands those values
  # into a full list.  So [1-5] becomes [1, 2, 3, 4, 5]
  vlan_ranges_loop_2 = {
    for k, v in local.vlan_ranges_loop_1 : "${v.key1}_${v.key2}" => {
      allocation_mode = v.allocation_mode
      description     = v.description
      key1            = v.key1
      key2            = v.key2
      role            = v.role
      vlan_list = length(regexall("(,|-)", jsonencode(v.vlan_range))) > 0 ? flatten([
        for s in v.vlan_split : length(regexall("-", s)) > 0 ? [for v in range(tonumber(
          element(split("-", s), 0)), (tonumber(element(split("-", s), 1)) + 1)
        ) : tonumber(v)] : [s]
      ]) : v.vlan_split
    }
  }

  # Loop 3 will take the vlan_list created in Loop 2 and expand this out to a map of objects per vlan
  vlan_ranges_loop_3 = flatten([
    for k, v in local.vlan_ranges_loop_2 : [
      for s in v.vlan_list : {
        allocation_mode = v.allocation_mode
        description     = v.description
        key1            = v.key1
        role            = v.role
        vlan            = s
      }
    ]
  ])

  # And lastly loop3's list is converted back to a map of objects
  vlan_ranges = { for k, v in local.vlan_ranges_loop_3 : "${v.key1}_${v.vlan}" => v }

  # End of Local Loops
}

After the locals manipulates the input values the data can then be looped through with the resources

resource "aci_vlan_pool" "vlan_pools" {
  for_each    = local.vlan_pools
  alloc_mode  = each.value.allocation_mode
  description = each.value.description
  name        = each.key
  name_alias  = each.value.alias
}

resource "aci_ranges" "vlans" {
  depends_on = [
    aci_vlan_pool.vlan_pools
  ]
  for_each     = local.vlan_ranges
  description  = each.value.description
  alloc_mode   = each.value.allocation_mode
  from         = "vlan-${each.value.vlan}"
  to           = "vlan-${each.value.vlan}"
  role         = each.value.role
  vlan_pool_dn = aci_vlan_pool.vlan_pools[each.value.key1].id
}

There might be easier ways but after thinking about this for almost a year this was the only way I was able to solve it, after learning a lot more about Terraform over the last year.