[SOLVED!] Modify Values for ZipMap while Transforming Map to new Keys

I thought this would be easy :frowning:

Goal is to transform this map:

accounts = {
  "acct-key-1" = {
    "billingcode" = "sys"
    "future-key" = "SOME-UNIQUE-VALUE-1"
    "private-attribute-1" = "fee"
    "private-attribute-2" = "foe"
  }
  "acct-key-2" = {
    "billingcode" = "sys"
    "future-key" = "SOME-UNIQUE-VALUE-2"
    "private-attribute-1" = "fie"
    "private-attribute-2" = "fum"
  }
}

into this map:

goodness = {
  "SOME-UNIQUE-VALUE-1" = {
      "billingcode" = "sys"
      "acct-key" = "acct-key-1" 
  }
  "SOME-UNIQUE-VALUE-2" = {
      "billingcode" = "sys"
      "acct-key" = "acct-key-2"
  }
}

As you can see, there are three tasks going on:

  1. Create the new map using a guaranteed-unique attribute (future-key) as the key for the new map. That’s not a problem, see code below.
  2. Insert the old key as an attribute into the new map
  3. Remove some named attributes (private-attribute-n) from the new map

Per code below, ZipMap seems to be the way to go partway there. Using ZipMap I get a partial result like this:

partial = {
  "SOME-UNIQUE-VALUE-1" = {
    "billingcode" = "sys"
    "future-key" = "SOME-UNIQUE-VALUE-1"
    "private-attribute-1" = "fee"
    "private-attribute-2" = "foe"
  }
  "SOME-UNIQUE-VALUE-2" = {
    "billingcode" = "sys"
    "future-key" = "SOME-UNIQUE-VALUE-2"
    "private-attribute-1" = "fie"
    "private-attribute-2" = "fum"
  }
}

with a few things wrong with it:

  • old key not inserted as an attribute
  • private attributes not removed
  • old key still present. Don’t really care about that one!

It seems that “all” I need to do is to modify local.newvalues to:

  • Insert the old key as an attribute, and
  • Remove the unwanted values

And I’ve tried nearly every variant of nested for…in loops I could think of and find on the web, with no success at all, not even enough to show.

The problem seems to be that local.newvalues is a tuple, and methods to modify tuples are few and far between.

The logic I’ve tried to implement would go like:

for each object in local.newvalues
get the corresponding old key from keys(accts) and insert it
look at other attributes and skip or remove them if they match one of the private keys

The code is simple and goes like this:

locals {
    # get source map
    accts = jsondecode(file("${path.module}/question.json"))
    newkeys = values(local.accts)[*].future-key
    newvalues = values(local.accts)
    partial = zipmap(
      local.newkeys, local.newvalues
    )
}

output "accounts" {
    value = local.accts
}

output "newkeys" {
    value = local.newkeys
}

output "newvalues" {
    value = local.newvalues
}

output "partial" {
    value = local.partial
}```

And the unfinished output like this:

Outputs:

accounts = {
“acct-key-1” = {
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-1”
“private-attribute-1” = “fee”
“private-attribute-2” = “foe”
}
“acct-key-2” = {
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-2”
“private-attribute-1” = “fie”
“private-attribute-2” = “fum”
}
}
newkeys = [
“SOME-UNIQUE-VALUE-1”,
“SOME-UNIQUE-VALUE-2”,
]
newvalues = [
{
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-1”
“private-attribute-1” = “fee”
“private-attribute-2” = “foe”
},
{
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-2”
“private-attribute-1” = “fie”
“private-attribute-2” = “fum”
},
]
partial = {
“SOME-UNIQUE-VALUE-1” = {
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-1”
“private-attribute-1” = “fee”
“private-attribute-2” = “foe”
}
“SOME-UNIQUE-VALUE-2” = {
“billingcode” = “sys”
“future-key” = “SOME-UNIQUE-VALUE-2”
“private-attribute-1” = “fie”
“private-attribute-2” = “fum”
}
}

To make code easy to reproduce, I put the initial map in a json file that the code reads and decodes; here it is as question.json:

{
    "acct-key-1": {
        "future-key": "SOME-UNIQUE-VALUE-1",
        "billingcode": "sys",
        "private-attribute-1": "fee",
        "private-attribute-2": "foe"
    },
    "acct-key-2": {
        "future-key": "SOME-UNIQUE-VALUE-2",
        "billingcode": "sys",
        "private-attribute-1": "fie",
        "private-attribute-2": "fum"
    }  
}

Ideas much appreciated

Is there some reason you can’t just do

  { for k, v in local.accounts : v["future-key"] =>
    {
      "billingcode" = v["billingcode"]
      "acct-key"    = k
    }
  }

?

No reason, that’s excellent! A case of not seeing the forest for the trees I think! Many thanks, very much appreciated!

The comprehension above is definitely the way to go (tested and in use!). Iterating over the newvalues = values(local.accts) and making changes remains of some academic interest for an undefined future problem, but for now we’re great to go!

Thanks for confirming, @rpattcorner!

As a starting point for your potential similar future problem, I think it is fair to say that for any sort of operation where you are filtering or projecting a collection (which includes maps) into another collection, for expressions will typically be involved. That’s the Terraform language’s primary construct for deriving one collection from another, and so tends to be at the core of most problems of that type.

We do sometimes need to use other functions in conjunction with the for expression, such as flatten to discard extra levels of nesting that can result from nested for expressions, but the for expressions are typically doing most of the heavy lifting.

Of course, you can always feel free to start a new topic here if you want to discuss the details of that later problem. :grinning: