Build a map with multiple nested for expressions controlled by a string split

Hi Everyone,

I have content from an csv file which gets imported in data & local. The imported data is a map which contains a tuple (because of group mode …) which contains an object with various values.
I use this content to create resources for Cisco ACI which works as expected.

But there is one thing which currently grills my brain.

I need to do a cisco epg (End Point Group) mapping where I have to map a dynamic amount of epgs to one server. The epgs a specified in the inner object as follows: "start:stop / start:stop / and so on.

Example: “1:3/40:40/50:51” meaning all eps starting from epg 1 to epg 3, epg 40 and all epgs starting from epg 50 to epg 51 → 1,2,3,40,50,51 should be mapped to one server which is specified by the key of the map (v.host_name)

data "local_file" "hosts-csv" {
  filename = "${path.module}/hosts.csv"
}

locals {
  # read the data source and create array with host_name as key
  hosts               = csvdecode(data.local_file.hosts-csv.content)
  hosts_profiles      = { for k, v in local.hosts : v.host_name => v... }
}

> type(local.hosts_profiles)
object({
    server1: tuple([
        object({
            acc_grp_policy: string,
            // further output omitted
        }),
    ]),
    server2: tuple([
        object({
            acc_grp_policy: string,
            // further output omitted
        }),
    ]),
})


output "variable" {
  value=local.hosts_profiles
}


  + variable = {
      + server1 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "1:3/40:40/50:51"
            },
        ]
      + server2 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test02 1-15"
              + cdp            = "cdp-enabled"
              + host_name      = "test02"
			  + epg            = "30:30
            },
        ]
    }

The epg value for server1 is reading like this: All eps starting from epg 1 to epg 3, epg 40 and all epgs starting from epg 50 to epg 51 → 1,2,3,40,50,51
The epg value for server2 is reading like this: All eps starting from epg 30 to epg 30 → 30

The desired result should be a map which I can use to iterate with for_each looking like this:

  + mapToIterate = {
      + server1:1 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "1"
            },
        ]
      + server1:2 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "2"
            },
        ]
      + server1:3 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "3"
            },
        ]
      + server1:30 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "30"
            },
        ]
      + server1:50 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "50"
            },
        ]
      + server1:51 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test01 1-10"
              + cdp            = ""
              + host_name      = "test01"
			  + epg            = "51"
            },
        ]		
      + server2:30 = [
          + {
              + acc_grp_policy = "auto"
              + accprtsel_name = "test02 1-15"
              + cdp            = "cdp-enabled"
              + host_name      = "test02"
			  + epg            = "30:30
            },
        ]
    }

I’ve managed to build a flattend list with a test value but unfortunately I’m not able to build a correct map with this syntax. In python I would pass the result from the first split as input to the next for loop but I’m grilling my brain how to do this in terraform.

My flattend test list looks like this:

epglist       = distinct(flatten([
  for entry in split("/", "1:3/40:40/50:51") : range(split(":", entry)[0],split(":", entry)[1] + 1)
]))

  + epglist_final = [
      + 1,
      + 2,
      + 3,
      + 40,
      + 50,
      + 51
    ]

Any help is highly appreciated.

Kind regards
Oliver

Hi @oliver.matt

Could you edit your post to use the preformatted text for your code blocks as it will make it a lot easier to read by honouring your indentation and not adding additional markdown formatting (such as the bullets). It will make it possible to copy/paste the code out of the post if people want to reproduce the scenario from your code.

You can use 3x ‘backtick’ characters on the line before and after your code blocks:

```
locals {
fred = “123”
}
```

Is then formatted as:

locals {
   fred = "123"
}

Or you can use the ‘<>’ button in the toolbar after highlighting code (This is good if you want to show something inline with your text.

I’ll take a look at this for you then, as it is hard to interpret quickly in its current format.

Thanks in advance.

Hi @ExtelligenceIT ,

thanks for letting me know. Code is reformated.

Thank you in advance.

1 Like

OK - My brain is hurting a bit now :slight_smile: but I think the below gives you what you need. It may not be the most efficient/direct approach but it’s all I can grok for now.

I have made some assumptions and comments based upon the structure of your hosts_profiles input object as it relates to the servers containing a tuple of an object, and that therefore it could contain multiple objects per server?

A more extensive example of the input file may need to be run through this to make sure it does give you what you expect.

You should be able to drop this into a main.tf in a bare directory and run terraform apply over it to play about.


locals {

  # Variable holding expected format of input after loading from file (Based upon your post)
  hosts_profiles = {
    server1 = [
      {
        acc_grp_policy = "auto"
        accprtsel_name = "test01 1-10"
        cdp            = ""
        host_name      = "test01"
        epg            = "1:3/40:40/50:51"
      },
      { # Note - as you show this as tuple of objects I have added another object but see below for merge constraints regarding duplicate EPGs/keys
        acc_grp_policy = "green"
        accprtsel_name = "test01-green 1-10"
        cdp            = ""
        host_name      = "test01"
        epg            = "4:7/45:45/49:51"
      }
    ]
    server2 = [
      {
        acc_grp_policy = "auto"
        accprtsel_name = "test02 1-15"
        cdp            = "cdp-enabled"
        host_name      = "test02"
        epg            = "30:30"
      }
    ]
  }


  # The nested for loops to create the map you want

  maptoiterate = merge( # <--- Merge will take the last unique item when there are duplicate keys in the maps - is this an issue?
    flatten([
      for server, server_values in local.hosts_profiles : [
        for value in server_values : {
          for epgval in distinct(
            flatten(
              [for entry in split("/", value.epg) : range(split(":", entry)[0], split(":", entry)[1] + 1)] # <--- Your original code
            )
            ) : "${server}:${epgval}" => { # <--- This is the key for the map and may be overwritten if there are duplicate EPGs across different policy maps in the list per server. But the key format could be extended with additional element to make it unique if the format is arbitrary 
            server         = server
            acc_grp_policy = value.acc_grp_policy
            accprtsel_name = value.accprtsel_name
            cdp            = value.cdp
            host_name      = value.host_name
            epg            = epgval
          }
        }
      ]
    ])
  ...)

}

# Write out the map to check it is as expected
output "maptoiterate" {
  value = local.maptoiterate

}



And here is the output (based on the above code):

Changes to Outputs:
  + maptoiterate = {
      + "server1:1"  = {
          + acc_grp_policy = "auto"
          + accprtsel_name = "test01 1-10"
          + cdp            = ""
          + epg            = 1
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:2"  = {
          + acc_grp_policy = "auto"
          + accprtsel_name = "test01 1-10"
          + cdp            = ""
          + epg            = 2
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:3"  = {
          + acc_grp_policy = "auto"
          + accprtsel_name = "test01 1-10"
          + cdp            = ""
          + epg            = 3
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:4"  = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 4
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:40" = {
          + acc_grp_policy = "auto"
          + accprtsel_name = "test01 1-10"
          + cdp            = ""
          + epg            = 40
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:45" = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 45
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:49" = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 49
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:5"  = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 5
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:50" = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 50
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:51" = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 51
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:6"  = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 6
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server1:7"  = {
          + acc_grp_policy = "green"
          + accprtsel_name = "test01-green 1-10"
          + cdp            = ""
          + epg            = 7
          + host_name      = "test01"
          + server         = "server1"
        }
      + "server2:30" = {
          + acc_grp_policy = "auto"
          + accprtsel_name = "test02 1-15"
          + cdp            = "cdp-enabled"
          + epg            = 30
          + host_name      = "test02"
          + server         = "server2"
        }
    }

Sorry it took a bit longer to reply than you may have hoped. Work always gets in the way of the fun stuff :smiley:

Hope that helps

Happy Terraforming

1 Like

Hi @ExtelligenceIT,

many thanks for your valued input. Since work and appointments are overwhelming right now it will take me 1 or 2 days to verify your code but your result already looks perfect to me.

Kind regards
Oliver