How to force a sequential execution of child modules in main.tf

Terraform
1

I am trying to setup a sequential execution of the following child modules in main.tf where add-membership depends on the output gitlab-project-ids list from add-gitlab-project but it is not working for me

module "add-gitlab-project" {
    source = "<module location>"
    modinput_token = var.tplinput_token
    modinput_input-file= var.tplinput_projects
    modinput_tags = var.tplinput_tags
    modinput_shared_groups = var.tplinput_shared_groups
}

module "add-membership" {
    source = "<module location>"
    modinput_token = var.tplinput_token
    modinput_input-file = var.tplinput_users
    modinput_access_level = var.tplinput_useraccess
    modinput_projectid = module.add-gitlab-project.gitlab-project-ids
}

terraform graph showed that both are still being loaded in parallel and since I use for_each in add-membership to loop through all the project_id and user pairs, it always error out with the complain that it does not know the number of project object to be used.

Each of the child module execute flawlessly of its own even with the for_each construct

2

Nest the module “add-membership” inside the “add-gitlab-project” module. If you pass the gitlab-projects-ids variable to the "add-membership module the graph will be built as requested.

With that said, I’d think that the output “module.add-gitlab-project.gitlab-project-ids” would set the graph as you expect.

3

Hi @mizunos,

What you’re describing doesn’t sound like a dependency problem but rather a problem with Terraform not having enough information to complete the plan. The for_each argument requires that the number of elements in the collection is known at plan time, but I suspect that your gitlab-project-ids is a set of strings that can’t be determined until the projects are created at apply time.

A common way to get a result like what you are looking for is to make the for_each value use keys that are controlled by your configuration rather than by the remote system. I can’t be sure exactly what that would be in your case with the subset of configuration you shared, but if the gitlab-project-ids is derived from a gitlab_project resource that also has for_each set then making the gitlab-project-ids value be a map derived from that resource might help:

output "gitlab-project-ids" {
  value = { for k, p in gitlab_project.example : k => p.id }
}

The important thing here is that the keys in the map (k in the above example) are values that are decided in the configuration, not in the remote GitLab server. That way Terraform will have a stable key to track the instances by, but will still give you access to the server-decided id value in each.value.

If the above isn’t feeling clear enough, I’d be happy to give some more detail if you can show the relevant parts of both of your modules – the output "gitlab-project-ids" block and whatever it depends on from the first module, and the variable "modinput_projectid" block and the resource that refers to var.modinput_projectid in the second module. With those, I should be able to give a more complete example.

4

Thanks for the detail explanation and I will try to apply your suggestion. To your question about the modules code, here they are:

the source for add-gitlab-project comes from the following

variable "modinput_token" {
  description = "authentication token to Terraform cloud"
}
variable "modinput_input-file" {
  description = "Projects to be created in a CSV file - first row identify all the field name to be used"

}

locals {
   project-list = csvdecode(file("${var.modinput_input-file}"))
}

resource "gitlab_project" "res_gl_project" {
## Required inputs
    lifecycle {
        prevent_destroy=true
    }
    for_each = {for inst in local.project-list:inst.name=>inst}
    name = each.value.name
    namespace_id = each.value.namespace_id
    description = each.value.description

output "gitlab_project_ids" {
  value = [for u in gitlab_project.res_gl_project:u.id]
}

And the source for gitlab-projectmembership is as followed:

variable "modinput_token" {
  description = "Token to authenticate to Terraform cloud"
}
variable "modinput_input-file" {
  description = "list of email of users that need to be added.  must be a csv format - See sample in repo"
}
variable "modinput_access_level" {
    description = " role that the users will have for a set of projects"
    default = "maintainer"
}

variable "modinput_projectid" {
  type = list(number)
}

locals {
   member-emails = csvdecode(file("${var.modinput_input-file}")) 
}

data "gitlab_user" "company_user" {
    for_each = {for inst in local.member-emails:inst.emails=>inst}
    email = each.value.emails
}

locals {
    userid = [for key, u in data.gitlab_user.company_user: {
       user-key = key
       user-id = u.id
    }]

    projectids = [for key, inst in var.modinput_projectid: {
       project-key = key
       project-id = inst
    }]

    membership = [
      for pair in setproduct(local.userid,local.projectids): {
         user = pair[0].user-id
         project = pair[1].project-id
      }]
}



resource "gitlab_project_membership" "toprojects" {
    for_each = {for sample in local.membership: "${sample.user}.${sample.project}" => sample}
    user_id = each.value.user
    access_level = var.modinput_access_level
    project_id = each.value.project
}

Any suggestion to address the for_each issue and dependency are greatly appreciated as I am struggling with this issuse for the last 2 days

5

Thanks for the extra information.

It looks like my earlier suggestion to switch the output to be a map would work. Here’s an adjustment using your actual resource name:

output "gitlab-project-ids" {
  value = { for k, p in gitlab_project.res_gl_project : k => p.id }
}

The second module will also need an adjusted declaration for the corresponding input variable:

variable "modinput_projectid" {
  type = map(number)
}

Then you should be able to use var.modinput_projectid, or some other map value derived from it with the same keys, as a for_each expression.