For_Each, Pass in multiple json files

Terraform
1

Hello!

I am trying to create a module for aws_cloudwatch_event_rule so I can create multiple rules which go to the same target.

This is my main.tf in module:

resource "aws_cloudwatch_event_rule" "event_rule" {
  for_each = {
  name        = var.cloudwatch_rule_name
  description = var.cloudwatch_rule_description

  event_pattern = var.event_pattern 
  }
}

resource "aws_cloudwatch_event_target" "sns" {

  for_each = aws_cloudwatch_event_rule.event_rule

  rule      = aws_cloudwatch_event_rule.event_rule[each.key].name
  target_id = "SendToSNS"
  arn       = aws_sns_topic.alerts.arn
}

And this the varible in variables.tf

variable "event_pattern" {
    description = "The event pattern described a JSON object"
    type = any
}

And lastly this is my module call

module "cloudwatch_event" {
    source = "./modules/cloudwatch-event"
    
    eventbridge_rule_name = "event_rule"
    eventbridge_rule_description = "rule_description"

    event_pattern = file("${path.module}/policy/cloudwatch_event_rules/ec2-health.json")     
}

I get the following error:

 Error: Missing required argument
│
│   with module.cloudwatch_event.aws_cloudwatch_event_rule.ec2_health,
│   on modules/eventbridge-event/main.tf line 23, in resource "aws_cloudwatch_event_rule" "event_rule":
│   23: resource "aws_cloudwatch_event_rule" "event_rule" {
│
│ "schedule_expression": one of `event_pattern,schedule_expression` must be specified

Any help would be much appreciated!

2

Hi @akrypto,

You seem to have written all of the arguments for the aws_cloudwatch_event_rule resource into the for_each object by mistake, and so from Terraform’s perspective that block is empty and so the provider is raising validation errors where you’ve not set some of the required arguments.

First then, let’s hoist those out to make it a valid single event rule, and then we can think about how to apply for_each to it as a separate step:

resource "aws_cloudwatch_event_rule" "event_rule" {
  name        = var.cloudwatch_rule_name
  description = var.cloudwatch_rule_description

  event_pattern = var.event_pattern 
}

This should now pass the rule that “one of event_pattern,schedule-expression must be specified”, because it defines event_pattern.

However, with that done I’m not sure what to do next because it isn’t clear to me from your example module definition what for_each would be repeating based on. Your module has a singular name cloudwatch_event and it takes only singular arguments for one rule name, one rule description, etc.

If your goal is to have all of the objects declared by the module be repeated based on some rule then you may find it easier to place the for_each on the module block itself, rather than on the resources inside:

module "cloudwatch_event" {
  source   = "./modules/cloudwatch-event"
  for_each = { /* ... */ }

  # ...
}

However I’m not sure what exactly to set that for_each to, based on the information you’ve shared so far. It might help to manually expand out the configuration as you would write it if you couldn’t use for_each here, to illustrate the pattern you’re trying to create, and then hopefully I can show you how to rework that into a systematic declaration of multiple objects with a single module block.

3

Hi @apparentlymart,

Thanks for the response, much appreciated!

I’ve managed to get the event patterns created by keeping the for_each in the resource like this…

resource "aws_cloudwatch_event_rule" "event_rule" {
  name        = var.cloudwatch_rule_name
  description = var.cloudwatch_rule_description

  event_pattern = var.event_pattern 
}

and doing the following with the module call.

module "cloudwatch_event" {
    source = "./modules/cloudwatch-event"
    
    event_rule = { 
        "ec2-health" = {
            "name" = "ec2_scheduled_stop"
            "description" = "EC2 stop scheduled"
            "rule" = {
                "source": ["aws.health"],
                "detail-type": ["AWS Health Event"],
                "detail": {
                "service": ["EC2"],
                "eventTypeCategory": ["scheduledChange"],
                "eventTypeCode": ["AWS_EC2_INSTANCE_STOP_SCHEDULED"]
                }
            }
        }
        "ec2-health2" = {
            "name" = "ec2_scheduled_test"
            "description" = "EC2 stop scheduled"
            "rule" = {
                "source": ["aws.health"],
                "detail-type": ["AWS Health Event"],
                "detail": {
                "service": ["EC2"],
                "eventTypeCategory": ["scheduledChange"]
                }
            }
        }            
    }    
}

However what I was trying to do was have these 2 event patterns in 2 json files and loop over them instead on defining the pattern in my code.

Something along the lines of event_rule = file("${path.module}/policy/eventbridge_event_rules/ec2-health.json")

4

Hi @akrypto,

The file function will return the contents of the file as a string containing the raw JSON syntax, so that alone isn’t sufficient to get a data structure you can manipulate in Terraform, but you can combine it with jsondecode to get a Terraform value corresponding to the JSON data structure.

I’m not sure I follow exactly where in your examples you’d like the JSON data to be used, but I see that the attribute name in your module’s event_rule object matches the example JSON filename you showed, and so perhaps this is in the direction you were hoping for:

module "cloudwatch_event" {
  source = "./modules/cloudwatch-event"

  event_rule = {
    "ec2-health" = jsondecode(file("${path.module}/policy/eventbridge_event_rules/ec2-health.json"))
    "ec2-health2" = jsondecode(file("${path.module}/policy/eventbridge_event_rules/ec2-health2.json"))
  }
}

If those two JSON files contain data structures equivalent to the ones you showed in your module "cloudwatch_event" block then Terraform will produce the same result as before.

1 Like
5

Hi @apparentlymart,

This is exactly what I was looking for! Awesome, thank you! Your help is much appreciated.

6

Hi @all,

check my multiple YAML and JSON file extractor. It will make your life easier from now on:

Cheers,
Levon