Multiple For_Each / Json Data manipulation single data set

Terraform
1

Burning out on this, hoping to see if someone has a better idea or direction. To start, I first collect my data from using http Data Source, the response is json. I then perform jsondecode , in locals like so:

locals {
  payload = jsondecode(data.http.Selected.body)
}

I am then able to use this data to create topics / subscriptions against the following resources azurerm_servicebus_topic, azurerm_servicebus_subscription using a for_each expression that looks something like this:

resource "azurerm_servicebus_topic" "azure_asb_topics" {
  for_each   = { for t in local.payload.publishes : t.topicName => t }
  name       = each.value.topicName
}

resource "azurerm_servicebus_subscription" "azure_asb_subscriptions" {
  for_each   = { for s in local.azureresourceapi_json.subscriptions : s.subscriptionName => s }
  name       = each.value.subscriptionName
}

My challenge is when it comes to creating azurerm_servicebus_subscription_rule the data I have for filters is a tuple of two elements for each individual subscription. So I am trying to apply another for_each but I cannot figure out the best way to pull the elements that are unique to each subscription by doing something like

resource "azurerm_servicebus_subscription_rule" "azure_asb_subscriptions_rule" {
  for_each            = { for r in local.azureresourceapi_json.subscriptions : r.filters => r }
  name                = each.value.filters.name
  resource_group_name = data.azurerm_servicebus_namespace.Selected.resource_group_name
  namespace_name      = data.azurerm_servicebus_namespace.Selected.name
  topic_name          = each.value.topicName
  subscription_name   = each.value.subscriptionName
  filter_type         = "SqlFilter"
  sql_filter          = each.value.filters.value
}

But I get an error r.filters is tuple with 2 elements, Im sure I either need to massage this data differently but I cant determine the best solution for this. I was leaning towards a flatten or a map that only has the values needed I can then iterate over… Any recommendation is much appreciated.

In closing locals.payload data looks like this in terraform console output:

{
  "appName" = "topic-v1.0.0"
  "publishes" = [
    {
      "topicName" = "topic-v1"
    },
  ]
  "subscriptions" = [
    {
      "autoDeleteOnIdle" = "P0DT0H5M0S"
      "deadLetteringOnFilterEvaluationExceptions" = "true"
      "deadLetteringOnMessageExpiration" = "true"
      "defaultMessageTimeToLive" = "P10DT0H0M0S"
      "duplicateDetectionHistoryTimeWindow" = "P0DT0H10M0S"
      "enableBatchedOperations" = "false"
      "filters" = [
        {
          "action" = "test"
          "name" = "mySqlFilter"
          "type" = "SqlFilter"
          "value" = "scope = 'autotest'"
        },
        {
          "action" = null
          "name" = "envfilter"
          "type" = "SqlFilter"
          "value" = "env = 'dev02'"
        },
      ]
      "lockDuration" = "P0DT0H5M0S"
      "maxDeliveryCount" = "2147483647"
      "requiresSession" = "false"
      "status" = "active"
      "subscriptionName" = "subscription-dev02"
      "topicName" = "topic-v1"
    },
    {
      "autoDeleteOnIdle" = "P0DT0H5M0S"
      "deadLetteringOnFilterEvaluationExceptions" = "true"
      "deadLetteringOnMessageExpiration" = "true"
      "defaultMessageTimeToLive" = "P10DT0H0M0S"
      "duplicateDetectionHistoryTimeWindow" = "P0DT0H10M0S"
      "enableBatchedOperations" = "false"
      "filters" = [
        {
          "action" = "test"
          "name" = "mySqlFilter"
          "type" = "SqlFilter"
          "value" = "scope = 'autotest'"
        },
        {
          "action" = null
          "name" = "envfilter"
          "type" = "SqlFilter"
          "value" = "env = 'dev01'"
        },
      ]
      "lockDuration" = "P0DT0H5M0S"
      "maxDeliveryCount" = "2147483647"
      "requiresSession" = "false"
      "status" = "active"
      "subscriptionName" = "subscription-dev01"
      "topicName" = "topic-v1"
    },
  ]
}
2

Hi @ctj3,

If I’m following correctly, it seems like you need to have one azurerm_servicebus_subscription_rule object per each pair of subscription and filter. Your idea of using flatten is my first idea too, because this seems to fit the pattern described in Flattening nested structures for for_each.

I think you could construct the intermediate flattened data structure like this:

locals {
  subscription_filters = flatten([
    for sub in locals.payload.subscriptions : [
      for filt in sub.filters : {
        subscription = sub
        filter       = filt
      }
    ]
  ])
}

This resulting data structure is a little redundant because it’ll include both a filter and the entire subscription (including all of the subscription’s filters again) for each element, but in practice that shouldn’t be a big deal because your subsequent for_each usage will ignore those redundant parts:

resource "azurerm_servicebus_subscription_rule" "azure_asb_subscriptions_rule" {
  for_each = {
    for sf in local.subscription_filters : "${sf.subscription.subscriptionName}:${sf.filter.name}" => sf
  }

  name                = each.value.filter.name
  resource_group_name = data.azurerm_servicebus_namespace.Selected.resource_group_name
  namespace_name      = data.azurerm_servicebus_namespace.Selected.name
  topic_name          = each.value.subscription.topicName
  subscription_name   = each.value.subscription.subscriptionName
  filter_type         = "SqlFilter"
  sql_filter          = each.value.filter.value
}

This will declare instances with addresses like this:

  • azurerm_servicebus_subscription_rule.azure_asb_subscriptions_rule["subscription-dev02:mySqlFilter"]
  • azurerm_servicebus_subscription_rule.azure_asb_subscriptions_rule["subscription-dev02:envfilter"]
  • azurerm_servicebus_subscription_rule.azure_asb_subscriptions_rule["subscription-dev01:mySqlFilter"]
  • azurerm_servicebus_subscription_rule.azure_asb_subscriptions_rule["subscription-dev01:envfilter"]
2 Likes
3

This is solid. Thank you so much for the example!