Iterate over groups of objects while incrementing values

[cfiske]

Hi all,

I’m trying to iterate over sets of grouped objects, but I also need an incremental offset for each item within a group. It seems like this is a use case for count but then I don’t feel like I can easily discover the resource instance key from within other resource definitions. I can’t quite get my head around how to iterate these in a way that makes them easy to associate the related resources.

So if I have roughly this:

my_map = {
  group1 => [
    { name = "host1", port = "443" },
    { name = "host2", port = "443" },
    { name = "host3", port = "443" }
  ]
  group2 => [
    { name = "host4", port = "5432" },
    { name = "host5", port = "5432" },
    { name = "host6", port = "5432" }
  ]
}

I need one load balancer per group, which is straightforward:

resource "aws_lb" "my_lb" {
  for_each = keys(var.my_map)

  ...
}

But I need the client to be able to pick which host by giving them different listeners + target groups.

resource "aws_lb_listener" "listener_per_host" {
  for_each = ???

  port = each.value.group_offset_plus_increment_value
  ...
}

So I can make a list of objects like:

merged_list = [
  { group = "group1", name = "host1", port = 443 },
  { group = "group1", name = "host2", port = 443 },
  { group = "group1", name = "host3", port = 443 },
  { group = "group2", name = "host4", port = 5432 },
  { group = "group2", name = "host5", port = 5432 },
  { group = "group2", name = "host6", port = 5432 }
]

and iterate over that, but then I have a collision on port number for the listeners. I need to end up with something like:

merged_list = [
  { group = "group1", name = "host1", port = 443, listen = 2000 },
  { group = "group1", name = "host2", port = 443, listen = 2001 },
  { group = "group1", name = "host3", port = 443, listen = 2002 },
  { group = "group2", name = "host4", port = 5432, listen = 3000 },
  { group = "group2", name = "host5", port = 5432, listen = 3001 },
  { group = "group2", name = "host6", port = 5432, listen = 3002 }
]

How would I create a unique offset per group, plus add 1 to the offset for each instance in the group?

Thanks in advance!

[cfiske]

So I was ultimately able to solve it, first by rearranging my source data a bit, then using index() to get incremental object numbers.

 locals {
    port_step  = 1000
    port_start = 5

    group_list = ["group1", "group2"]
    my_map = {
      web1 = {
        group = "group1"
        name  = "host1"
        port  = 443
      }
      web2 = {
        group = "group1"
        name  = "host2"
        port  = 443
      }
      web3 = {
        group = "group1"
        name  = "host3"
        port  = 443
      }
      db1 = {
        group = "group2"
        name  = "host4"
        port  = 5432
      }
      db2 = {
        group = "group2"
        name  = "host5"
        port  = 5432
      }
      db3 = {
        group = "group2"
        name  = "host6"
        port  = 5432
      }
    }

    group_map = {
      for group in local.group_list :
      group => [
        for key, obj in local.my_map : {
          name = obj.name
          port = obj.port
        } if group == obj.group
      ]
    }

    merged_map = {
      for group in keys(local.group_map) :
      group => [
        for obj in local.group_map[group] : {
          name   = obj.name
          port   = obj.port
          listen = (local.port_start + index(keys(local.group_map), group)) * local.port_step + index(local.group_map[group], obj)
        }
      ]
    }
  }

In real code I can compile group_list from the group data in my_map but this was just for test purposes. Now merged_map looks like this:

{
    group1 = [
        {
            listen = 5000
            name   = "host1"
            port   = 443
          },
        {
            listen = 5001
            name   = "host2"
            port   = 443
          },
        {
            listen = 5002
            name   = "host3"
            port   = 443
          },
      ]
    group2 = [
        {
            listen = 6000
            name   = "host4"
            port   = 5432
          },
        {
            listen = 6001
            name   = "host5"
            port   = 5432
          },
        {
            listen = 6002
            name   = "host6"
            port   = 5432
          },
      ]
  }

Hopefully this is helpful for anyone else trying to do this. I am open to any feedback on how I could do this more efficiently as well.