Google provider: google_monitoring_alert_policy label extractors not working as expected

Terraform Providers
1

I am using v5.20.0 of the Google provider, and cannot seem to get label_extractors to work as advertised by the Google API documentation (search for “LogMatch” slightly below that link). The docs suggest it should be possible to include labels extracted from the logs in the notification messages generated by the alert. I have had no luck. I am curious if others have had success.

My goal is to provide the actual log message in the notification to allow a quick triage without requiring multi-step navigation. My belief is that I should be able to extract field from the message and then reference them in the content attribute as follows:

Request Log Example

Within my google_monitoring_alert_policy, I have included:

  conditions {
    display_name = "Request Log Error"
    condition_matched_log {
      filter = <<EOT
resource.type="cloud_run_revision"
resource.labels.service_name="${var.service_name}
log_name="projects/${var.gcp_project_id}/logs/run.googleapis.com%2Frequests"
severity>"WARNING"
EOT
      label_extractors = {
        request   = "EXTRACT(jsonPayload.httpRequest)"
      }
    }
  }
  documentation {
    mime_type = "text/markdown"
    subject   = "$${resource.label.service_name} Request Log Error"
    content   = <<EOT
## $${resource.label.service_name} Request Log Error

### Summary

- **Project**: $${resource.project}
- **Cloud Run Service**: $${resource.label.service_name}
- **Cloud Run Revision**: $${resource.label.revision_name}
- **Alert Policy**: $${policy.display_name}
- **Condition**: $${condition.display_name}
- **Request Details**: $${log.extracted_label.request}
EOT
  }

Application Log Example

Within my google_monitoring_alert_policy, I have included:

  conditions {
    display_name = "Application Log Error"
    condition_matched_log {
      filter = <<EOT
resource.type="cloud_run_revision"
resource.labels.service_name="${var.service_name}
log_name="projects/${var.gcp_project_id}/logs/run.googleapis.com%2Fstdout"
severity>"WARNING"
EOT
      label_extractors = {
        log  = "EXTRACT(textPayload)"
      }
    }
  }
  documentation {
    mime_type = "text/markdown"
    subject   = "$${resource.label.service_name} Application Log Error"
    content   = <<EOT
## $${resource.label.service_name} Application Log Error

### Summary

- **Project**: $${resource.project}
- **Cloud Run Service**: $${resource.label.service_name}
- **Cloud Run Revision**: $${resource.label.revision_name}
- **Alert Policy**: $${policy.display_name}
- **Condition**: $${condition.display_name}
- **Log Message**: $${log.extracted_label.log}
EOT
  }

Result

In both cases I find that the variable for the extracted label is not interpolated. It is just passed through in the output as:

${log.extracted_label.request}

or:

${log.extracted_label.log}

For now I’ve dropped the effort and am instead including instructions for finding the log entry through the GCP console. I’m grateful for any leads that can help me make this a better experience.

Thanks!

1 Like