Numbering objects adding calculated counter number without sorting

Terraform
1

Hello HashiCorp forum! Currently I’m facing an issue that unfortunately I could not resolve on my own yet.

What I’m trying to is adding a calculated key:value pair to an existing map of objects without disturbing the order. For expressions unfortunately does rearrange the order of objects.

Current code tested with:

locals {
  peerings = {
    d = {
      name = "d"
    }
    c = {
      name = "c"
    }
    r = {
      name = "r"
    }
    b = {
      name = "b"
    }
    m = {
      name = "m"
    }
    p = {
      name = "p"
    }    
    a = {
      name = "a"
    }
  }
  peerings_counter = {
    for key, net in local.peerings : key => merge({ counter = index(keys(local.peerings), key) + 1 }, net)
  }
}

The output of that code:

object1 = {
  + a = {
      + count = 1
      + name  = "a"
    }
  + b = {
      + count = 2
      + name  = "b"
    }
  + c = {
      + count = 3
      + name  = "c"
    }
  + d = {
      + count = 4
      + name  = "d"
    }
  + m = {
      + count = 5
      + name  = "m"
    }
  + p = {
      + count = 6
      + name  = "p"
    }
  + r = {
      + count = 7
      + name  = "r"
    }
}

Output Im looking for:

object1 = {
  + a = {
      + count = 7
      + name  = "a"
    }
  + b = {
      + count = 4
      + name  = "b"
    }
  + c = {
      + count = 2
      + name  = "c"
    }
  + d = {
      + count = 1
      + name  = "d"
    }
  + m = {
      + count = 5
      + name  = "m"
    }
  + p = {
      + count = 6
      + name  = "p"
    }
  + r = {
      + count = 3
      + name  = "r"
    }
}

Does anyone have and idea on how to solve that? I’m open for any changes as long as the solution does not include adding a manual counter value and the local must be able usable as a for_each reference in a module block.
Thanks!

2

If the order is important, you’ll probably need to use lists. Still trying to understand the use case though. Could you elaborate on it?

3

In Terraform (and indeed many programming languages) map-like types are explicitly unordered, so this is not possible - as the order has already been thrown away by the parser, before you as the programmer ever get your hands on the data.

You could possibly do something like:

locals {
  peerings = [
    {
      name = "d"
    },
    {
      name = "c"
    },
    {
      name = "r"
    },
    {
      name = "b"
    },
    {
      name = "m"
    },
    {
      name = "p"
    },
    {
      name = "a"
    },
  ]
  peerings_counter = {
    for index, net in local.peerings : net.name => merge({ count = index + 1 }, net)
  }
}

but I really don’t recommend it as all the counts will move around if items are added/removed from the list, which usually causes unwanted side-effects.

1 Like
4

In a TF repo for GCP network configuration are many projects stored inside similar local objects and their count is currently maintained by hand.
Every 32. (and multiple of 32) project or object in that local list invokes a creation of a new Cloud Router and Transit Gateway and much more.

The counter is currently added manually and the project objects are not sorted in any way and should not get sorted, because changing the order of the counter values would cause the projects VPN tunnels to get assigned new Cloud Routers and lead to a downtime which is not worth the change.

I was trying to automate the count to prevent errors and gaps in the counter values which caused issues in the past.

5

But if you automate it, and then want to decommission a project, isn’t that going to cause lots of other projects to shift counter value, causing lots of unwanted reconfiguration?

6

Yes you are right but the more important thing is the initial order that is already established should not get mixed up in the process.
What will happen in future, when we’re removing a project in the middle of the order is negligible because it happens rather rarely. Most of the times projects at the end of the list are destroyed

7

This problem of managing the relationships between arbitrary integers and different objects unfortunately tends to arise a lot in network-related configurations where it’s a physical constraint that all systems have a unique numerical IP address.

When I’ve encountered this requirement before I’ve typically solved it by using a list as the input, so that the ordering is preserved as others have said, but then decided on a convention for “skipping” a particular element in the list so that it’s possible to deallocate an old assignment without upsetting everything after it.

One example of that which is published in a public place is the hashicorp/subnets/cidr module, which tries to encapsulate the problem of generating an IP addressing plan based on a data structure describing the requirements, with the specific numbers then selected automatically.

In that module the convention for skipping a removed allocation is too set its name to null, in which case it still gets its address space allocated but it’s excluded from the final table of CIDR prefixes by name.

I’m not sure I fully understood the use-case discussed in this thread but it seems like a similar pattern might work here.

1 Like