Extract key value from json and populate list in Terraform

Terraform
1 
  "Events": [
    {
      "Topic": "value1"
    },
    {
      "Topic": "value2"
    },
    {
      "Topic": "value3"
    }]
}```

with the above json. I'd like to extract the key values for each Topic and use them in a list in terraform template in this format ["value1","value2","value3"]. I've tried using the jsondecode but getting errors. Perhaps not doing it correct. Also use this but to no avail.... ```data "external" "topic" {
  program = ["jq", ".dictionary_name", "files.json"]
  query   = {}

}```

What is the best way to get this done? I'd imagine it's the jsondecode that I need to figure out but just not sure.   thank you.
2

Hi @rbankole,

I’m not sure I fully understood your goal but I think you want to find a list of just the Topic property values from those event objects. The JSON you shared doesn’t seem complete but assuming that the JSON file contains a top-level JSON object where Events is one of its properties then you could do that using something like the following:

locals {
  raw_data     = jsondecode(file("${path.module}/example.json"))
  event_topics = local.raw_data.Events[*].Topic
}

local.raw_data is a representation of the contents of that JSON file in the Terraform type system. Because the JSON file contains a JSON object, the result is a Terraform object value.

We can then use Terraform expressions to work with that object value. In local.event_topics I used a splat expression to concisely select the Topic attribute from each of the nested objects in the local.raw_data.Events sequence.

2 Likes
3

Thank you so much for this…very helpful.

4

Ok I really appreciate the help but now I’ve run into another issue with the terraform expression for subsequent nested objects.

{
“Topics”: [{
“Name”: “emailsent”,
“DefaultMessageTimeToLive”: “14 days”,
“DuplicateDetectionHistory”: “30 seconds”,
“EnablePartitioning”: true,
“EnableBatchedOperations”: true,
“RequireDuplicateDetection”: true,
“Subscribers”: [{
“Name”: “events-logger”,
“MaxDeliveryCount”: 100,
“DefaultMessageTimeToLive”: “14 days”,
“LockDuration”: “30 seconds”,
“ForwardTo”: “events-logger-catch-all”,
“EnableBatchedOperations”: true,
“EnableDeadLetteringOnMessageExpiration”: true,
“EnableDeadLetteringOnFilterEvaluationError”: true
}]
}]
}

In my tf moduels, i’m able to call the the Topic names with this expression **topics = local.topics.Topics[*].Name** and that works but now need to call the values for Subscriber name,maxdeliverycout,defaultmessage etc. I call them with this forward_to = local.topics.Topics[*].Subscribers[*].ForwardTo but get an error with the syntax.

What is the proper way to call those values for the keys under Subscribers in my module?

5

Hi @rbankole,

Once you get past simple cases of selecting a single attribute for each object it can help to switch from splat syntax to the more general for expression.

A splat expression is, in some sense, a shorthand for a common case of a for expression. The example I originally shared of selecting the Topic attribute could be written instead as a for expression, like this:

locals {
  event_topics = [for e in local.raw_data.Events : e.Topic]
}

The [ ] brackets around the expression indicate that you want the result to be a sequence. Terraform will then evaluate e.Topic for each of the elements in local.raw_data.Events to produce a sequence of those results, which should be the same as the splat expression.

But the for expression has more capabilities: it can construct both mappings and sequences and it can evaluate any arbitrary expression for each element.

How best to deal with your nested lists will depend on exactly what data structure you’re hoping to get. For example, if you wanted to build a two-level map structure with the first level being topic name keys and the second level being subscriber keys then you could do that this way:

locals {
  topic_subscriber_forward_to = {
    for t in local.topics.Topics : t.Name => {
      for s in t.Subscribers : s.Name => s.ForwardTo
    }
  }
}

With the example data structure you shared, that would produce something like this:

{
  "emailsent": {
    "events-logger": "events-logger-catch-all"
  }
}

As another example, if you just wanted to find all of the distinct values of ForwardTo without caring which topics or subscribers each one was associated with, you could combine for expressions with the flatten and toset functions:

locals {
  distinct_forward_to = toset(flatten([
    for t in local.topics.Topics : [
      for s in t.Subscribers : s.ForwardTo
    ]
  ]))
}

This one will result in a set with a single element "events-logger-catch-all" in your example.

Hopefully those examples are useful. If you’re not sure how to adapt those examples to achieve your goal, I can perhaps provide a more specific example if you can show me what you’d like your result to look like if given the example input you shared in your question.

1 Like
6

A post was split to a new topic: Select particular map element based on environment name