Nested loop (using github provider)

I’m trying to use the github provider to configure repository settings for an organisation. What I’d like to achieve is to specify a simple configuration (e.g. using variables) in which I can list all our repos, the settings for those repos, users for those repos etc.

Here’s the problem. The github_repository_collaborator allows me to specify collaborators for a repo, but only one at once. The username attribute doesn’t accept lists, it only supports single usernames. I thus obviously need to use recursion. However, I also need to do this for multiple repos, which effectively means double recursion.

This was all under the assumption that I structure my inputs to the config file something like this:

repo_configs = {
    repository_1 = {
        github_repository = {
            allow_merge_commit = true
            allow_squash_merge = false
            allow_rebase_merge = false
        },
        github_branch_protection = {
            required_pull_request_reviews = {
                required_approving_review_count = 1
            }
        }
    },
    repository_2 = {
        github_repository = {
            allow_merge_commit = true
            allow_squash_merge = true
            allow_rebase_merge = false
        },
        github_branch_protection = {
            required_pull_request_reviews = {
                required_approving_review_count = 2
            }
        }
    }
}    

If I can’t double recurse then I’m going to have to change the inputs (maybe split up) and flatten to something like

repo_users = [
    {
        repo = repository_1
        username = bobsmith
        permission = "pull"
    },
    {
        repo = repository_1
        username = johndoe
        permission = "push"
    },
    {
        repo = repository_2
        username = bobsmith
        permission = "push"
    }
]

Yuck.

What’s the best way to achieve my goal whilst keeping input data DRY and concise?

Hi @HormyAJP,

The main tools for dealing with this sort of situation are for expressions and the flatten function. With these two features, we can transform collections and flatten out nested collections for use with the repetition constructs like resource for_each.

You didn’t show how you want the collaborators to be represented in your configuration structure but I’m going to assume a variable defined like this for the sake of example:

variable "repositories" {
  type = map(object({
    allow_merge_commit = bool
    allow_squash_merge = bool
    allow_rebase_merge = bool
    branch_protection = object({
      required_approving_review_count = number
    })

    # Map from username to permission level,
    # like {"bobsmith" = "pull"}.
    collaborator_permissions = map(string)
  }))
}

To transform this into a flat set of (repository, username, permission) structures we’ll use flatten with some for expressions:

locals {
  repo_collaborators = flatten([
    for rn, r in var.repositories : [
      for un, pn in r.collaborator_permissions : {
        repo       = rn
        username   = u
        permission = pn
      }
    ]
  ])
}

resource "github_repository_collaborator" "example" {
  # Each instance must have a unique key, so we'll
  # construct them by concatenating the repository
  # name and the username.
  for_each = {
    for rc in local.repo_collaborators :
    "${rc.repo}:${rc.username}" => rc
  }

  repository = each.value.repo
  username   = each.value.username
  permission = each.value.permission
}

This will produce resource instances with addresses like github_repository_collaborator.example["repository_1:bobsmith"], allowing Terraform to correlate any changes to the collaborator permissions maps with the appropriate resource instances for ongoing updates.

Hi @apparentlymart,

How would you import existing resources when the module to create them is going to use a for_each with a concatenated name?

Is there a better option than having to do a “terraform state mv” for everything?

If you are talking about objects that aren’t currently tracked in Terraform at all, you should be able to import them to addresses with string keys directly with terraform import:

terraform import 'aws_instance.example["foo"]' i-abc123

(Note that the resource address is in single quotes to prevent a POSIX-style shell from interpreting the quotes around foo. On Windows, backslash escaping is needed instead. PowerShell adds additional complications.)

If you have existing objects that are already tracked by Terraform using numeric indices because they were created using count, then indeed terraform state mv is the only way to tell Terraform how to map the existing indices to the new keys, because Terraform cannot infer those relationships automatically.

terraform state mv 'aws_instance.example[0]' 'aws_instance.example["foo"]'