Get map of maps of objects from data source returning single object

Terraform
1

data source aws_iam_policy returns an object with attrubutes about a policy, passing a policy arn. Using a configuration like this:

locals{
  maps_of_policy_arns = {
    managed1 = {
      pol1 = "arn:aws:iam::aws:policy/IAMFullAccess",                                 # Allow IAM resource management
      pol2 = "arn:aws:iam::aws:policy/AmazonEC2FullAccess",                           # Allow EC2 resource management
      pol3 = "arn:aws:iam::aws:policy/AmazonS3FullAccess",                            # Allow S3 resource management
    }
    managed2 = {
      pol1 = "arn:aws:iam::aws:policy/AmazonRDSFullAccess",                           # Allow RDS management
      pol2 = "arn:aws:iam::aws:policy/AWSDirectoryServiceFullAccess",                 # Allow DirectoryService management  
    }
  }
}

data "aws_iam_policy" "managed1" {

  for_each = local.maps_of_policy_arns.managed1
  arn = each.value
}

data "aws_iam_policy" "managed2" {

  for_each = local.maps_of_policy_arns.managed2
  arn = each.value
}

I can get e.g. from data.aws_iam_policy.managed1 a map of objetcs like this (I’m interested to inline policy part - policy) :

{
  "pol1" = {
    "arn" = "arn:aws:iam::aws:policy/IAMFullAccess"
    ...
    "policy" = <<-EOT
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["iam:*",
          "organizations:DescribeAccount",
          ...
          "Resource": "*"
        }
      ]
    }
    EOT
  }
  "pol2" = {
    "arn" = "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
    ...
    "policy" = <<-EOT
    {
    ...
    EOT
   ...
}

I’d like to avoid static behavior imposing to create a data source for each map of policy arns (e.g. managed1, managed2, managedN), making more dynamic and DRY code, but as this data source returns a single object I cannot go over a map of objects using for_each.

It is possible to get a maps og objects like this ?

  maps_of_policy_documents = {
    managed1 = {
      "pol1" = {
        "arn" = "arn:aws:iam::aws:policy/IAMFullAccess"
        ...
        "policy" = <<-EOT
        ...
        EOT
      }
      "pol2" = {
        "arn" = "arn:aws:iam::aws:policy/AmazonEC2FullAccess"
        ...
        "policy" = <<-EOT
        ...
        EOT
      }
      ...
    }
    managed2 = {
      "pol1" = {
      ...
      }
      ...
    }
  }

e.g. using some dynamic approach like flatten([…]) and local for loop with expression (without using static filtering condition on key name etc.) ?

2

Hi @Roxyrob!

Based on what you’ve described here it seems like the two-level heirarchy of policy groupings is something only relevant to your Terraform configuration and not something that the remote IAM API is aware of. If that’s true then I think you’re on the right track with flatten, with the idea of producing just a flat set of policy ARNs that you can request using instances of the data resource and then project into the map structure again afterwards.

Because you’re starting with a map of maps rather than lists we’ll need to project this into a list of lists in order to use flatten with it, which we can do with a for expression like this:

locals {
  policy_arns = toset(flatten([
    for group in local.maps_of_policy_arns : [
      for arn in group : arn
    ]
  ]))
}

I’m not sure if this is important for your use-case but notice that because we’re flattening these all into a single set it would be possible for the same policy ARN to appear in multiple maps, in which case it’ll only appear once in this set (because sets are unordered collections of unique values) and so will only be fetched once when we use this with the data resource, which would look like this:

data "aws_iam_policy" "managed" {
  for_each = local.policy_arns

  arn = each.value
}

This data resource will now have one instance per distinct ARN, and will be a map of objects where the map keys are the ARNs. However, to get the final result you wanted there’s one more step of melding the policy objects with the original map of maps data structure, looking up the policy results by their ARNs:

locals {
  policies = tomap({
    for gk, group in local.maps_of_policy_arns : gk => tomap({
      for pk, arn in group : pk => {
        arn    = arn
        policy = data.aws_iam_policy.managed[arn].policy
      }
    })
  })
}

Assuming that I managed to write that all out without any typos (I’m afraid I don’t currently have AWS credentials handy to test with), I think local.policies will then be the map of map of objects that you were hoping for, where each of the inner objects has arn and policy attributes.

3

Hi @apparentlymart, thank you.

I already made the aws_iam_policy data source as map with arns as keys starting from a flattened list of the arns, but I couldn’t found the right for loop to “melding the policy objects with the original map of maps data structure” as you defined here.

The uniqueness of the arns in the flattened set is not an issue for my use case: I need to mitigate policies for role limits imposed by AWS, so all these policies will be attached to a single role, so it is almost a requirements not having duplicated policies.

I can now replicate this solution for every role/policies mapping, with a more DRY, simple and flexible code, merging policies statements in less policy objects (to mitigate policies for role limit) and add new groups if present ones will reach another AWS limit (policy size).

local.policies was exactly what I’m looking for.