Having difficulty using two different for loops in the same resource

thedarkwriter · January 9, 2020, 6:22pm

Hi there,

Terraform v0.12.18

provider.google v3.4.0
provider.google-beta v3.4.0

I am trying to create several service accounts and map them to multiple roles in GCP.

I’m having an issue getting my second loop to be honored (if that’s possible).

variable "roles_for_admins" {
  default = {
    "iam" = "roles/resourcemanager.projectIamAdmin"
    "kubernetes" = "roles/container.admin" 
    "storage" = "roles/storage.admin"
    "datastore" = "roles/datastore.owner"
    "googleappengine" = "roles/appengine.appAdmin"
    "computevpc" = "roles/compute.admin"
    "cloudfunctions" = "roles/cloudfunctions.admin"
    "cloudscheduler" = "roles/cloudscheduler.admin"
    "cloudtasks" = "roles/cloudtasks.admin"
    "memorystore" = "roles/redis.admin"
    "serverlessvpcconnector" = "roles/vpcaccess.admin"
  }
}

variable "admins" {
   default = {
     "joesmith" = "jsmith"
     "alicebrown" = "abrown"
     "anotherone" = "aone"
   }
}

resource "google_service_account" "create-serviceaccounts" {
  for_each = var.admins
  account_id   = each.value
  display_name = "This service account is for ${each.value} to manage things"
}

resource "google_project_iam_member" "grant-deployer-roles-to-users" {
  for_each = var.roles_for_admins
  role = each.value
  member = "serviceAccount:${[for admin in google_service_account.create-serviceaccounts: admin.email]}"
  depends_on = [google_service_account.create-serviceaccounts]
}

The problem loop is the member = "serviceAccount:${[for admin in google_service_account.create-serviceaccounts: admin.email]}"

The error looks like it doesn’t like how I’m trying to call that for loop (possibly because the member attribute expects only a string or because this loop is different than the original for_each)

Example error
Error: Invalid template interpolation value

  on line 51, in resource "google_project_iam_member" "grant-deployer-roles-to-users":
  51:   member = "serviceAccount:${[for admin in google_service_account.create-serviceaccounts: admin.email]}"
    |----------------
    | google_service_account.create-serviceaccounts is object with 6 attributes

Cannot include the given value in a string template: string required.

What I’m trying to get working is having one block to create all the users and one block to iteratively add in all the roles to each user.

apparentlymart · January 9, 2020, 8:02pm

Hi @thedarkwriter,

It sounds like what you need here is to have one google_project_iam_member object for every unique combination of elements from var.roles_for_admins and var.admins.

Terraform’s setproduct function is useful for this sort of use-case, and its documentation includes an example using AWS networks and subnets, which I think can adapt reasonably easily to your use-case here:

locals {
  admin_role_memberships = [
    # all of the distinct combinations of values from the two variables
    for pair in setproduct(values(var.admins), values(var.roles_for_admins)) : {
      account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]]}"
      role    = pair[1]
    }
  ]
}

resource "google_project_iam_member" "admins" {
  for_each = {
    for m in local.admin_role_memberships : "${m.account} ${m.role}" => m
  }

  role   = each.value.role
  member = each.value.account
}

The for expression in the for_each projects the list of objects into a map of objects as required for for_each, concatenating together the two values to make a unique identifier for each instance. The instances will therefore have addresses like google_project_iam_member.admins["abrown roles/storage.admin"].

thedarkwriter · January 9, 2020, 9:38pm

Hey @apparentlymart, thanks so much for the help.

It doesn’t quite go through, I’m getting a similar error as last time

Error: Invalid template interpolation value

  on ../../modules/infrastructure/modules/auth/service_account_roles.tf line 21, in locals:
  21:       account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]]}"
    |----------------
    | google_service_account.create-serviceaccounts is object with 3 attributes
    | pair[0] is "abrown"

Cannot include the given value in a string template: string required.

google_service_account.create-serviceaccounts is object with 3
attributes is the part I think is getting borked (probably trying to return the account (email, name, unique_id))

apparentlymart · January 10, 2020, 2:35am

Oh, whoops! Sorry, I made an editorial error when I was writing that part and forgot to include the attribute name:

      account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]].email}"

Notice that it now has .email on the end of that reference to google_service_account.create-serviceaccounts, so that the result will be a single string.

Although these two errors have similar text, they have slightly different causes: in your case it was because you tried to interpolate a list (the result of the for expression) into a template, while in my case I advertently tried to use an entire google_service_account object where I intended to refer only to its email attribute. Either way though, anything that appears in a ${ ... } sequence must always produce a result that can be converted to a string, because template interpolation is a string concatenation operation.

thedarkwriter · January 13, 2020, 2:30am

As you explained it, it makes total sense why that would work, but something still isn’t right. I’ve tried mucking with the config for a while but I’m not getting a good result. If I put in what you recommend:

account = "serviceAccount:${google_service_account.terraform-admins[pair[0]].email}"

I get:
Error: Invalid index

  on ../../modules/infrastructure/modules/auth/service_account_roles.tf line 21, in locals:
  21:       account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]].email}"
    |----------------
    | google_service_account.create-serviceaccounts is object with 3 attributes
    | pair[0] is "abrown"

The given key does not identify an element in this collection value. <<<THIS IS THE CHANGE

I am guessing the “3” attributes it is talking about are the three users in the variable. If I add a user to variable “admins”, the terraform run will tell me “4” attributes. I would expect the google_service_account to have 6 attributes.

What are your thoughts? Thank you again for all your help!

apparentlymart · January 13, 2020, 6:15pm

Hi @thedarkwriter!

I’m afraid my weak knowledge of GCP is causing me to guess a bit and to be unable to test directly, and it looks like I got some of the details wrong.

Perhaps it’d be better to take a step back and talk about the concepts involved in that example rather than me just continuing to try to tweak it until it “works”. Hopefully then you can combine this knowledge about Terraform with your existing knowledge about GCP and find the solution you need yourself.

With that said, let’s take a look at the expression that is now indicating an error:

google_service_account.create-serviceaccounts[pair[0]]

We can see in the error message that pair[0] here is "abrown". That’s coming from values(var.admins) in the setproduct call: I wrote it to take the values from that map, rather than the keys, and so we are getting "jsmith", "abrown", "aone" there.

However, the for_each for google_service_account.create-serviceaccounts is also var.admins, and so google_service_account.create-serviceaccounts here behaves as a map using the keys from var.admins: joesmith, alicebrown, anotherone. Therefore looking up "abrown" there fails: there is no element with that key in the map.

Therefore I think in order to fix it we need to use the keys from var.admins in the setproduct call, which will then give us the keys we need for that map lookup:

    for pair in setproduct(keys(var.admins), values(var.roles_for_admins)) : {

By using keys instead of values, pair[0] will take on the values joesmith, alicebrown, and anotherone, which should then be compatible with the google_service_account.create-serviceaccounts map.

The value of local.admin_role_memberships will then be as follows:

[
  {
    account = "serviceAccount:jsmith@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/resourcemanager.projectIamAdmin"
  },
  {
    account = "serviceAccount:jsmith@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/container.admin"
  },
  # (many more jsmith roles omitted)
  {
    account = "serviceAccount:abrown@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/resourcemanager.projectIamAdmin"
  },
  {
    account = "serviceAccount:abrown@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/container.admin"
  },
  # (many more abrown roles omitted)
  {
    account = "serviceAccount:aone@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/resourcemanager.projectIamAdmin"
  },
  {
    account = "serviceAccount:aone@PROJECT_ID.iam.gserviceaccount.com"
    role    = "roles/container.admin"
  },
  # (many more aone roles omitted)
 
]

So when we use this one in the for_each for google_project_iam_member.admins we need to use another for expression to convert it into a map, as required by for_each:

  for_each = {
    for m in local.admin_role_memberships : "${m.account} ${m.role}" => m
  }

This then meets the expectations of for_each a map with one element per desired resource instance, each with a unique key, and with each element value containing all of the data we need to populate the arguments of the resource type. each.value in that resource block is, therefore, an object with role and account attributes as we see in the local.admin_role_memberships value above.

thedarkwriter · January 15, 2020, 3:23am

Hey @apparentlymart,
You’ve really helped me here, I totally appreciate it. With your guidance I was able to get this working and found that my errors were a bit sneakier than I realized. Once I got this config in (I’ll share below), I realized that I was getting errors creating the local.admin_role_memberships object because if all the users weren’t already created by google_service_account.create-serviceaccounts then the plan and apply would bomb like so:

Error: Invalid for_each argument

  on ../../modules/infrastructure/modules/auth/service_account_roles.tf line 57, in resource "google_project_iam_member" "admins":
  57:    for_each = local.admin_role_mapping

This line here: account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]].email}" would be populated with “” strings and when the role mappings would start, it would try to add role = "" and member = "" (and further complain of duplicates if I were trying to add more than 1 user that didn’t exist)

So, if I first ensured that I ran a:
terraform apply --target=module.infrastructure.module.auth.google_service_account.create-serviceaccounts"

It would create all the accounts. Then I could run a normal terraform apply -auto-approve and it would properly iterate over all the for_eaches and add the roles to the users. I tried to get creative with using ternary operators, flattens, depends_on, and a few other things to trick locals into skipping any not-yet-created user.email resource but I just couldn’t get it to work. For now, I’ll just have to create the users on target, then run everything. Deletes work just fine

variable "roles_for_gcp" {
  default = {
    "iam"                    = "roles/resourcemanager.projectIamAdmin"
    "kubernetes"             = "roles/container.admin"
    "storage"                = "roles/storage.admin"
    "datastore"              = "roles/datastore.owner"
    "googleappengine"        = "roles/appengine.appAdmin"
    "computevpc"             = "roles/compute.admin"
    "cloudfunctions"         = "roles/cloudfunctions.admin"
    "cloudscheduler"         = "roles/cloudscheduler.admin"
    "cloudtasks"             = "roles/cloudtasks.admin"
    "memorystore"            = "roles/redis.admin"
    "serverlessvpcconnector" = "roles/vpcaccess.admin"
  }
}

locals {
  admin_role_memberships = [
    for pair in setproduct(keys(var.admins), values(var.roles_for_gcp)) : {
      account = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]].email}"
      role    = pair[1]
    } 
  ]
}

locals {
  admin_role_mapping =  {
       for m in local.admin_role_memberships : "${m.account} ${m.role}" => m 
   } 
}

# If any of these are not created, the run will bomb 
variable "admins" {
   default = {
     "joesmith" = "jsmith"
     "alicebrown" = "abrown"
     "anotherone" = "aone"
   }
}

resource "google_service_account" "create-serviceaccounts" {
  for_each     = var.admins
  account_id   = each.key
}

resource "google_project_iam_member" "admins" {
   for_each = local.admin_role_mapping  
 
   role   = each.value.role
   member = each.value.account 

}

Thank you again for all your help!

apparentlymart · January 15, 2020, 4:26pm

Ahh, yes… I didn’t consider that this email argument would be decided during apply and thus isn’t a good key to use in for_each.

If you’d like to spend a little more time smoothing that out, the important pattern is to write i so that the names you’ve chosen in the configuration are what are used in the key, rather than the names chosen by the provider itself. In your case, that would be either the keys or the values from var.admins, assuming both of those are always constants in the configuration. You can still use the .email attribute in the value of the for_each expression; the key being known during plan is the important thing.

I think I’d start that by changing local.admin_role_memberships to make sure you have both values available:

locals {
  admin_role_memberships = [
    for pair in setproduct(keys(var.admins), values(var.roles_for_gcp)) : {
      username = pair[0]
      account  = "serviceAccount:${google_service_account.create-serviceaccounts[pair[0]].email}"
      role     = pair[1]
    } 
  ]
}

With this, you can use m.username instead of m.account in the key part of your admin_role_mapping, but still retain the full account attribute for use via each.value.account inside your resource "google_project_iam_member" block.

Note that making this change now will change your instance keys and thus Terraform will see it as destroying the existing memberships and creating new ones. On the plus side though, their addresses will then be a little more compact: google_project_iam_member.admins["joesmith roles/resourcemanager.projectIamAdmin"]. You could also use the keys of your var.roles_for_gcp instead of the values to make it even more compact, but I’ll have to leave that part as an exercise.

apparentlymart · April 25, 2022, 4:26pm

2 posts were split to a new topic: “Invalid reference” for app-gen-eng-sandbox reference

Topic		Replies	Views
Pass For_each GCP Service account email output to another Module to bind roles Terraform	3	684	January 31, 2023
More than on Service account when each on has more than one role using terraform HCP Terraform hcp-terraform	0	364	February 8, 2021
Unable to associate multiple iam user with the custom iam role in gcp Terraform	8	1060	July 21, 2022
Want Multiple Provider and for_each Terraform	2	68	July 24, 2024
Assistance with Looping through information Terraform	0	235	August 13, 2021

Having difficulty using two different for loops in the same resource

Related topics