Terraforming AWS Glue triggers for multiple jobs and/or multiple conditions

Terraform Providers AWS
1

Hi all,

I’m having troubles with terraforming some AWS Glue triggers. I have a use case for a trigger to be able to execute more than one job in parallel, or (for a conditional trigger) to monitor more than one job with different conditional logic and then start the next jobs.

I can do it by using trigger resource, but this is not in line with the current Terraform setup, which uses modules. The problem is that I can not define the actions nor the conditions dynamically within the module.

This is what works, but I can’t use:

    resource "aws_glue_trigger" "example_multi_trigger_2" {
      name = "example-multi-trigger-2"
      type = "CONDITIONAL"
      workflow_name = module.glue_jobs_second_wf.aws_glue_workflow_id

      actions {
        job_name = module.glue_job_3.aws_glue_job_id
      }

      actions {
        job_name = module.glue_job_4.aws_glue_job_id
      }

      predicate {
        conditions {
          job_name = module.glue_job_1.aws_glue_job_id
          state    = "SUCCEEDED"
        }
        logical = "ANY"
        conditions {
          job_name = module.glue_job_2.aws_glue_job_id
          state    = "SUCCEEDED"
        }
      }
    }

This is the type of a workflow I’m aiming at:
image

And this is the current setup, implementing a module for the Glue Trigger:

module main.tf:

    locals {
        base_name = "dev-${var.trigger_name}-glue-trigger"
    }

    resource "aws_glue_trigger" "this" {
      name          = local.base_name
      type          = "CONDITIONAL"
      workflow_name = var.workflow_name

      actions {
        job_name = var.action_job_name
      }

      predicate {
        conditions {
          job_name = var.watched_job_name
          state    = var.watched_event
        }
      }
    }

variables.tf:

    variable "trigger_name" {
      description = "Name of the trigger"
    }

    variable "workflow_name" {
      description = "Name of the workflow to which trigger belongs"
      default     = ""
    }

    variable "action_job_name" {
      description = "Name of the job which should be invoked by trigger"
    }

    variable "watched_job_name" {
      default = ""
    }

    variable "watched_event" {
      default = ""
    }

    variable "enabled" {
      description = "Trigger enabled"
      default     = true
    }

and the way I currently can implement it, with a possibility to declare watched_job_name and action_job_name only once:

    module "trigger_job_2" {
      source           = "./modules/conditional_trigger"

      trigger_name     = "job-2"
      watched_job_name = module.glue_job_1.aws_glue_job_id
      watched_event    = "SUCCEEDED"
      action_job_name  = module.glue_job_2.aws_glue_job_id
      workflow_name = module.glue_jobs_main_wf.aws_glue_workflow_id
    }

I’m thinking there must be a way to achieve it, but so far I wasn’t been able to find it.
Any help from the TF community will be much appreciated!

Thanks!