Terraform and Datadog JSON variables

Terraform Providers
1

I have a question. We are working on a SRE type project, generalizing datadog resources with variables. We want to have a user only have to edit the variables for their env and not have to touch the main code. We are using Terraform with Datadog provider and using the JSON option.

So for this datadog monitor, we want a variables file for all the options. As you can see below, is an example of a datadog monitor we exported and using the JSON option.

What is the best method to add variables for each option, for example name, tags, thresholds, and etc.

I tried using this method, but running into issues. I just wanted to see if their are other option, maybe the tfvars file way?

variable "s3_bucket_size_monitor_json_monitor_name" {
  description = "Monitor Name"
  default     = "S3 Bucket Size Monitor - MaC"
  type        = string
}

Another example:

variable "s3_bucket_size_monitor_json_dd_common_tags" {
  type = map(string)
  default = {
    "atm_id"       = "aa34drt65"
    "xmatters"      = "false"
    "severity"      = "low"
    "env"           = "DEV"
    "auto_incident" = "false"
    "episode_view"  = "Incident Managed"
  }
}

Datadog Monitor:

resource "datadog_monitor_json" "s3_bucket_size_monitor_json" {
  monitor = <<-EOF
{
	"name": "PROM S3 Bucket Size Monitor",
	"type": "query alert",
	"query": "max(last_5m):max:aws.s3.bucket_size_bytes{bucketname:prom-performance-test} >= 1100000000000",
	"message": {{bucketname.name}} in region {{region.name}} has exceeded the threshold with the value of {{value}}.\n\n---\n\nSee [Datadog's S3 dashboard](https://app.datadoghq.com/screen/integration/101/aws-s3?tpl_var_bucketname={{bucketname.name}}) for a historical view\non the bucket.  \n\nView in [AWS Console](https://console.aws.amazon.com/s3/buckets/{{bucketname.name}}/?region={{region.name}}&tab=overview) \n\n---\n\n{{#is_alert}} @SRE size over 1 TB {{/is_alert}}\n\n{{#is_warning}} @SRE S3 Bucket size over 838 GB {{/is_warning}} ",
	"tags": [
		"atm_id:aa34drt65",
		"env:dev",
		"episode_view:Incident Managed",
		"auto_incident:false",
		"severity:low",
		"xmatters:false"
	],
	"options": {
		"thresholds": {
			"critical": 1100000000000,
			"warning": 900000000000
		},
		"notify_audit": false,
		"require_full_window": false,
		"notify_no_data": false,
		"renotify_interval": 0,
		"include_tags": true,
		"evaluation_delay": 900,
		"notification_preset_name": "hide_handles",
		"silenced": {}
	},
	"priority": 3,
	"restricted_roles": null
}
EOF
}

Please let me know what you think

Devon

2

We started doing it this way:

S3.tf

options": {
		"thresholds": {
      "critical": "${var.s3_bucket_size_monitor_json_options_thresholds_critical}",
      "warning": "${var.s3_bucket_size_monitor_json_options_thresholds_warning}"
    },
		"notify_audit": "${var.s3_bucket_size_monitor_json_options_notify_audit}",
		"require_full_window": "${var.s3_bucket_size_monitor_json_options_require_full_window}",
		"notify_no_data": "${var.s3_bucket_size_monitor_json_options_notify_no_data}",
		"renotify_interval": "${var.s3_bucket_size_monitor_json_options_renotify_interval}",
		"include_tags": "${var.s3_bucket_size_monitor_json_options_include_tags}",
		"evaluation_delay": "${var.s3_bucket_size_monitor_json_options_evaluation_delay}",
		"notification_preset_name": "${var.s3_bucket_size_monitor_json_options_notification_preset_name}",
		"silenced": "{${var.s3_bucket_size_monitor_json_options_silenced}}"

Variables:

variable "s3_bucket_size_monitor_json_monitor_message" {
  type        = string
  description = "Monitor Message"
  default     = "@webhook-SPLUNK-WEBHOOK \n\nThe size for bucket {{bucketname.name}} in region {{region.name}} has exceeded the threshold with the value of {{value}}.\n\n---\n\nSee [Datadog's S3 dashboard](https://app.datadoghq.com/screen/integration/101/aws-s3?tpl_var_bucketname={{bucketname.name}}) for a historical view\non the bucket.  \n\nView in [AWS Console](https://console.aws.amazon.com/s3/buckets/{{bucketname.name}}/?region={{region.name}}&tab=overview) \n\n---\n\n{{#is_alert}} @teams-MaC_Development S3 Bucket size over 1 TB {{/is_alert}}\n\n{{#is_warning}} @teams-MaC_Development S3 Bucket size over 838 GB {{/is_warning}}"
}
variable "s3_bucket_size_monitor_json_options_thresholds_critical" {
  type = string
  description = "Monitor Options Thresholds Critical"
  default = "1100000000000"
}
variable "s3_bucket_size_monitor_json_options_thresholds_warning" {
  type = string
  description = "Monitor Options Thresholds Warning"
  default = "900000000000"
}
variable "s3_bucket_size_monitor_json_options_notify_audit" {
  type = string
  description = "Monitor Options Notify Audit"
  default = "false"
}
variable "s3_bucket_size_monitor_json_options_require_full_window" {
  type = string
  description = "Monitor Options Require Full Window"
  default = "false"
}

So far so good, except for two issues we ran into. One is the “message” variable and the other is the Tags. The TF provider does not like the “/n” within the message and I have not figured out how do to the tags yet.

Please take a look and let me know if anyone has any better ideas.

Devon