How to get the distinct map value for the list obect to create resources

Terraform
1

Hi All, I need to create resource group from the json input which has list of maps as bellow, I am getting an duplicate key error as one of the entries for a resourcegroup is identical… how can i get the distinct map object?
“Storage”: [
{
“Resourcegroup_Name”: “agp-tf-rg”,
“Region”: “Central India”,
“Storagename”: “tfstorageacctcomp”,
“Containername”: “tstate-dev-cont”
},
{
“Resourcegroup_Name”: “agp-sa-rg”,
“Region”: “Central India”,
“Storagename”: “agpcgdsacomp”,
“Containername”: “agpcgdcont”
},
{
“Resourcegroup_Name”: “agp-tf-rg”,
“Region”: “Central India”,
“Storagename”: “rfstorageacctcomp”,
“Containername”: “rtstate-dev-cont”
}
]

2

Hi @janapb,

It seems like your first and third object are totally identical and so there isn’t any way to distinguish them with just the information given here.

Perhaps by “distinct map object” you mean that you’d like to just discard that duplicate object and keep only one of them? If that’s your goal then one way to achieve that would be to convert your list into a set of objects first, because the set data structure has the characteristic that each distinct value is either in the set or not in the set… there is no sense of the values being in any particular order or appearing more than once.

You can use the toset function to convert from a list to a set, silently discarding any duplicate values. You could then convert the result into a map however you already tried to do it and know that all of the duplicate objects were removed.

Note that toset on a list of objects will use the entire object value to decide about whether a value is a duplicate, rather than just the Resourcegroup_Name key, and so it’d still be invalid to have two different objects with the same name, but it’d be acceptable to have two objects of the same name as long as all of their other attributes are also equal.

3

The first and third entry, Storage account and Container name is unique, I will give a try with toset function and will update on the progress.

4

Ahh yes, I spent a long time staring at those trying to understand what was different about them but the difference was too subtle to notice until you pointed it out to me. Sorry for the misleading answer.

The answer I gave won’t help when the objects don’t exactly match, so here’s a different answer: you can group them by the key, so that there can be more than one for each key:

locals {
  grouped = {
    for obj in var.objects : obj.Resourcegroup_Name => obj...
  }
}

In the above example the important part is the ... modifier after obj, which tells Terraform to create a list of objects for each key rather than just a single object. That will then create you a data structure like this:

{
  "agp-tf-rg" = [
    {
      "Resourcegroup_Name" = "agp-tf-rg"
      "Region" = "Central India"
      "Storagename" = "tfstorageacctcomp"
      "Containername" = "tstate-dev-cont"
    },
    {
      "Resourcegroup_Name" = "agp-tf-rg"
      "Region" = "Central India"
      "Storagename" = "rfstorageacctcomp"
      "Containername" = "rtstate-dev-cont"
    },
  ]
  "agp-sa-rg" = [
    {
      "Resourcegroup_Name" = "agp-sa-rg"
      "Region" = "Central India"
      "Storagename" = "agpcgdsacomp"
      "Containername" = "agpcgdcont"
    },
  ]
}

Notice that the “agp-tf-rg” element has two sub-items now. This data structure probably isn’t sufficient for what you need for for_each, but you can use it as a starting point for various other transformations that might get you what you need.

For example, if you want to just take the first object associated with each key, you can project it like this:

locals {
  firsts = {
    for k, objs in local.grouped : k => objs[0]
  }
}

This constructs a new map where each element is only the first object from each of the lists in local.grouped, and so you’d effectively be ignoring the second “agp-tf-rg” object entirely.

5

I apologize for not giving clear information initially when i posted. I was creating the resources in phases where i was creating storage accounts and its Resource groups as a starting point to store everything locally. The idea is to park all the storage accounts in this phase, I was using the count and unique resource groups so everything went well as expected. The issue raise to me when i wanted to add additional Storage account within one of the resource group. Hope its clear now.

I will give a try with the above logic, Thanks for your support again.