[SOLVED] HTTP + jq returns empty values

Terraform Providers
1

Hello,

We’re trying to create grafana alerts in an existing grafana setup. I’m using an HTTP provider to query the grafana API for the existing folders, and an JQ provider to query the folder UID.

Info about our account:

  • Terraform Cloud - Free Tier
  • Terraform CLI version - 1.3.6

The providers are setup like this:

terraform {
  required_providers {
    grafana = {
      source  = "grafana/grafana"
      version = ">= 1.34.0"
    }
    httpclient = {
      source = "dmachard/http-client"
      version = "0.0.3"
    }
    jq = {
      source = "massdriver-cloud/jq"
      version = "0.2.1"
    }
  }
}

The folderUID is queried from the API like this:

data "httpclient_request" "folders" {
  url = "https://grafana.domain.com/api/folders"
  username = "testuser"
  password = "Password01"
  request_headers = {
    Accept: "application/json"
    Content-Type: "application/json"
    #Authorization: "Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxx=="
  }
}

# Query the JSON for the customer folder UID
data "jq_query" "folderuid" {
  data = data.httpclient_request.folders.response_body
  query = ".[]|select(.title | startswith(\"${var.customer_namespace}\"))|.uid"
}

And this data us used like this:

resource "grafana_rule_group" "errors_4xx" {
  
  name             = "${var.environment} - 4xx's"
  folder_uid       = jsondecode(data.jq_query.folderuid.result)
  interval_seconds = 60
  org_id           = 1
  
  rule {
    name           = "${var.environment} - 4xx's"
...

But when running this in Terraform Cloud, we always receive the following error:

│ Error: Error in function call
│ 
│   on ../../../shared/modules/grafana/alert_rules/__4xxs.tf line 6, in resource "grafana_rule_group" "errors_4xx":
│    6:   folder_uid       = jsondecode(data.jq_query.folderuid.result)
│     ├────────────────
│     │ while calling jsondecode(str)
│     │ data.jq_query.folderuid.result is ""
│ 
│ Call to function "jsondecode" failed: EOF.

We do not know why this is happening. Running the same code locally seems to work fine:

% terraform plan
data.httpclient_request.example: Reading...
data.httpclient_request.example: Read complete after 0s [id=https://grafana.control.journeyagency.travel/api/folders]
data.jq_query.folderuid: Reading...
data.jq_query.folderuid: Read complete after 0s [id=1674732582]

Changes to Outputs:
  + folderuid = "ZRPqrBKVz"

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

Any suggestion on why this is not working in Terraform Cloud?

2

I’ve been doing some more tests, and it seems that using a variable in the query breaks it. Using the following:

data "jq_query" "folderuid" {
  data = data.httpclient_request.folders.response_body
  #query = ".[]|select(.title | startswith(\"${var.customer_namespace}\"))|.uid"
  query = ".[] | select(.title | contains(\"nametolookfor\")) .uid"
}

Seems to work… any idea why using a variable in the query would break it?

  query = ".[]|select(.title | startswith(\"${var.customer_namespace}\"))|.uid"
3

I’ve found the issue. I was querying for startswith but not specifying the correct string. Changing the query to:

query = ".[]|select(.title | contains(\"${var.customer_namespace}\"))|.uid"

Works! The issue is that what I’m searching for does contain the customer_namespace, but doesn’t start with it, there’s more sting which when using startswith obviously causes it to fail.