Filter a list during construction

Terraform
1

I’m trying to filter a list of Azure locations to pick out the primary ones; primary location in this sense is the first location of a region pair that is in the list e.g. given a list [ 'westeurope', 'northeurope' ] the resulting list should be [ 'westeurope' ].

So to solve this in the general case, I need to look at the partially generated new list to see if the current value’s pair is already present and I can’t come up with a syntax/method that works.

The only hack I have is something like this…

    locals {
      pairs = {
        # Europe
        northeurope        = "westeurope"
        westeurope         = "northeurope"
    
        # UK
        uksouth            = "ukwest"
        ukwest             = "uksouth" 
    
        # Plus lots more ignored for brevity
      }
    
      # Just filter even indicies
      primaries = [ for x, index in var.locations : x if index / 2 = 0 ]
    }

But it requires that the items in the correct order i.e. would work for [ 'westeurope', 'northeurope', 'uksouth', 'ukwest' ] but would provide an incorrect response for [ 'westeurope', 'uksouth', 'northeurope', 'ukwest' ]

Here’s a table showing var.locations and the expected results for the different scenarios…

var.locations primaries
[ ‘westeurope’ ] [ ‘westeurope’ ]
[ ‘northeurope’ ] [ ‘northeurope’ ]
[ ‘westeurope’, ‘northeurope’ ] [ ‘westeurope’ ]
[ ‘northeurope’, ‘westeurope’ ] [ ‘northeurope’ ]
[ ‘westeurope’, ‘uksouth’, ‘northeurope’ ] [ ‘westeurope’, ‘uksouth’ ]
[ ‘westeurope’, ‘northeurope’, ‘uksouth’ ] [ ‘westeurope’, ‘uksouth’ ]
2

Hi @phatcher,

I think what I’m missing from your question here is what rule decides that westeurope is “more primary than” northeurope. From the data you’ve shared I only see bidirectional pairings – that northeurope and westeurope belong together – and not anything which defines westeurope as the “winner”.

Might it be helpful to instead present the “pairs” as ordered lists of two elements, like this?

locals {
  pairs = [
    ["westeurope", "northeurope"],
    ["uksouth", "ukwest"],
  ]
}

This now declares not only which names belong together in a pair but also gives an ordering within each pair. We might decide that the zeroth element of each of the pairs is the “primary”, and programmatically transform this into the more convenient data structure of a map that tells you the primary associated with a given location:

locals {
  primary_for_location = merge([
    for pair in local.pairs : tomap({
      pair[0] = pair[0]
      pair[1] = pair[0]
    })
  ]...)
}

What I did here was use a for expression to construct a list of maps where each map is shaped like this:

{
  "westeurope"  = "westeurope"
  "northeurope" = "westeurope"
}

Then I used the merge function to combine those all into a single map, using the ... symbol to expand the list into a series of separate arguments like merge expects, and thus producing a combined result like this (for my example input above):

{
  "westeurope"  = "westeurope"
  "northeurope" = "westeurope"
  "uksouth"     = "uksouth"
  "ukwest"      = "uksouth"
}

I think this lookup table now gives us the data needed to answer the question of whether to filter out a particular location from the input, per your requirement which I understand as: if both the primary and secondary of a pair are both present in the input, discard the secondary.

This now seems like a set-theory kind of problem and so let’s define var.locations as being a set of strings and build from there:

variable "locations" {
  type = set(string)
}

locals {
  selected_primaries = toset([
    for l in var.locations : l
    if local.primary_for_location[l] == l || !contains(var.locations, local.primary_for_location[l])
  ])
}

The condition here is, in English:

  • If the location is its own primary, OR
  • If the location’s primary isn’t also present in the input

local.selected_primaries should therefore be the output side of your table of examples.

1 Like
3

@apparentlymart Thanks for the response.

Since it’s a business decision as to which one region is primary for deployment (probably based on you or your customer’s locality), the choice of winner is the pair item with the lowest index in the original list, i.e. it’s equally valid for your primary region to be northeurope and your pair is westeurope or vice-versa.

For most resources e.g. App Service Plans, SQL Server etc you would deploy into both regions, but for things like storage accounts, your should change the SKU to make it region redundant (GRS, RA-GRS) and only deploy into the primary region - which is what I’m trying to make my modules do given a list of regions.

In an imperative language, when constructing a new list via iteration you could put a function to determine if a value or its pair was in the partially constructed list whereas hcl seems to be more declarative in its approach, hence my question :grinning: