Cycle dependency in terraform resources

Terraform Providers AWS
1

I want to create 2 aws_scheduler_schedule resources, one to start an instance and another to stop it.

Those schedulers must of course have the corresponding roles; here is how I go about it

# ---------------------------------------------------------------------------------------------------------------------
# GITHUB-RUNNER EC2 SCHEDULERS
# ---------------------------------------------------------------------------------------------------------------------

resource "aws_scheduler_schedule" "gh_runners_ubuntu_start" {
  for_each = var.gh_runners
  name     = format("%s-%s", "gh_runners_ubuntu_start", each.value.runner_id)

  flexible_time_window {
    mode = "OFF"
  }

  schedule_expression = each.value.start_time

  target {
    arn      = "arn:aws:scheduler:::aws-sdk:ec2:startInstances"
    role_arn = aws_iam_role.gh_runners_ubuntu_scheduler_role[each.key].arn

    input = jsonencode({
      InstanceIds = [
        module.gh_runners_ubuntu[each.key].id
      ]
    })
  }
}

resource "aws_scheduler_schedule" "gh_runners_ubuntu_stop" {
  for_each = var.gh_runners
  name     = format("%s-%s", "gh_runners_ubuntu_stop", each.value.runner_name)

  flexible_time_window {
    mode = "OFF"
  }

  schedule_expression = each.value.stop_time

  target {
    arn      = "arn:aws:scheduler:::aws-sdk:ec2:stopInstances"
    role_arn = aws_iam_role.gh_runners_ubuntu_scheduler_role[each.key].arn

    input = jsonencode({
      InstanceIds = [
        module.gh_runners_ubuntu[each.key].id
      ]
    })
  }
}

resource "aws_iam_role" "gh_runners_ubuntu_scheduler_role" {
  for_each = var.gh_runners
  name     = format("%s-%s", "gh_runners_ubuntu_scheduler_role", each.value.runner_name)

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "events.amazonaws.com"
        }
        Condition = {
          ArnEquals = {
            "aws:SourceArn" = [
              aws_scheduler_schedule.gh_runners_ubuntu_start[each.key].arn,
              aws_scheduler_schedule.gh_runners_ubuntu_stop[each.key].arn,
            ]
          }
        }
      }
    ]
  })
}

resource "aws_iam_role_policy" "gh_runners_ubuntu_scheduler_role_policy" {
  for_each = var.gh_runners
  name     = format("%s-%s", "gh_runners_ubuntu_scheduler_role_policy", each.value.runner_name)
  role     = aws_iam_role.gh_runners_ubuntu_scheduler[each.key].id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = ["ec2:StopInstances", "ec2:StartInstances"]
        Resource = module.gh_runners_ubuntu[each.key].arn
      }
    ]
  })
}

the plan fails as follows:

Error: Cycle: aws_scheduler_schedule.gh_runners_ubuntu_stop, aws_iam_role.gh_runners_ubuntu_scheduler_role, aws_scheduler_schedule.gh_runners_ubuntu_start

How can I go about it?

2

I don’t know much about AWS … so I’m just answering this from a pure Terraform perspective.

It looks like this is impossible to resolve - the issue is that there is no order in which Terraform can create these resources, as they mutually the require the ARNs of each other in their configuration.

The only way I can think of in which this configuration could come into being, would be for at least one of them to be created with a placeholder value, which was later updated to the real ARN to be referenced, to break the cycle… And this is not something Terraform supports.

I did wonder if there might be some kind of additional Terraform resource, which could attach the assume_role policy to the role… it is a fairly common pattern in Terraform when one piece of configuration needs to be applied to another… But in this case, the iam_role resource doesn’t seem to have any way to not specify the assume_role policy at creation time.

Hopefully this is some use in explaining why the error occurs… Maybe someone with more AWS-specific knowledge can help more.

1 Like
3

I am on par with what you replied.
I am working around this for the moment by removing the condition (makes the setup slightly less secure however)

4

This seems to be a long running issue.

5

You can often contruct an ARN instead of using a reference to a resource.

The format is well known (containing account ID, etc.) and is just a string.

6

I think this is the same question I answered on Stack Overflow earlier:

I won’t repeat my entire answer here, but the short answer is that this seems to be a consequence of the underlying AWS API design… both of the operations involved are designed under the assumption that the other one had already completed, which cannot possibly be true because one must come before the other.

Therefore the only way I can see to successfully use this underlying API is to use the documentation to predict what format one of the sets of ARNs will have and manually write the ARN ahead of time. The Terraform equivalent of that would be to write a Terraform expression that generates the predicted ARNs based on a rule, and so that’s what I illustrated in the answer on Stack Overflow.