Transitioning some dynamic `newrelic_dashboard` code to `newrelic_one_dashboard` resource

[aaa]

I have some intricate code, some of it alluded to in a past post where I used a dynamic { } block to build out custom widgets for the newrelic_dashboard resource, which will soon no longer work. We are required to move to the new newrelic_one_dashboard quite soon, because by the end of next month the newrelic_dashboard will stop working.

Root Module
In my current code for the newrelic_dashboard resource and building out dynamic widget { }s, customizable and extensible in terms of how many widget { }s you’d like, I have this code in my root module for passing in data mappings of the content I’d like for each individual widget { } to be created by the child module in its newrelic_dashboard (see section after this one):

# locals for passing customized/dynamic widgets data to the child module `nr_dashboard`
locals {
  # mappings for multiple widgets
  dynamic_widgets = [
    {
      title         = "Requests per minute"
      visualization = "billboard"
      entity_ids    = [data.newrelic_entity.my_application.application_id, ]
      nrql          = "SELECT rate(count(*), 1 minute) FROM Transaction WHERE appName = '${var.nr_entity_name}'"
      row           = 1
      column        = 1
    },
    {
...

Of course, the above is passed to the child module that creates the newrelic_dashboard via the module { } block:

module "newrelic_dashboard" {
    ...
     widgets       = local.dynamic_widgets
     ....

Current Child Module which Creates the newrelic_dashboard
In the child module which creates the dashboard, I have the code for the dashboard:

resource "newrelic_dashboard" "exampledash" {
  title = "New Relic Terraform Example"

  filter {
    event_types = [
        "Transaction"
    ]
    attributes = [
        "appName",
        "name"
    ]
  }

dynamic "widget" {
  for_each = var.widgets  # `widgets` is passed to us from Root module locals
  content {
    title = widget.value.title
    visualization = widget.value.visualization
    ....# other required values for a `widget { }`
    entity_ids  = try(widget.value.entity_ids, null)  # `try() is used for all attributes that are optional, such as `entity_ids`
    nrql  = try (widget.value.nrql, null)
    ....
    dynamic "metric" {
      for_each = try(widget.value.metric.name
      content {
        name = widget.value.metric.name
        valus = widget.value.metric.values
      }
    }
....

Issue with New Relic One Dashboard

The issue with the newrelic_one_dashboard is that widget isn’t the name of the argument anymore for using widgets. Each widget type now has it’s own custom name - prior to this the newrelic_dashboard kind of widget { } was defined by the widget { }s visualization argument - those new custom / pre-defined widget names being for example:

• widget_bar
• widget_billboard
• widget_bullet, etc

So the issue is, how do I easily convert from my old code for newrelic_dashboard to the newrelic_one_dashboard, given the change in how widgets are defined in the newrelic_one_dashboard resource?

Could I perhaps try to interpolate in the child module’s current dynamic "widget { } block and do something like have an array of widget suffixes like so:

suffixes = [ "bar", "billboard", "bullet" ]

and then have some type of surrounding iterative block for the above array - perhaps a for_each or other? - and interpolate each of the above values one by one into the dynamic "widget" { } block by changing said block to the below, or similar?:

dynamic "widget_${suffix}" { 
 .....

Just curious, but would be great if I could move my current code to the new newrelic_one_dashboard pretty seamlessly and keep my dynamic and extensible code for widgets.

That was a lot, thanks a lot to anyone in advance who reads/assists!

[apparentlymart]

Hi @aaa,

The best way I can think of to work with this new resource schema design is to first preprocess the list of widgets into a map of lists grouped by type, like this:

locals {
  widgets_by_viz = {
    for w in var.widgets : w.visualization => w...
  }
}

This is a for expression with the ... grouping mode enabled, and so the result will be a map of lists where the map keys are all of the distinct visualization types in the input.

You can then use that grouped list to dynamically produce the required blocks for each visualization type:

  dynamic "widget_billboard" {
    for_each = try(local.widgets_by_viz.billboard, [])
    content {
      # ...
    }
  }

  dynamic "widget_markdown" {
    for_each = try(local.widgets_by_viz.markdown, [])
    content {
      # ...
    }
  }

  # etc

I assume that the provider developers here chose to use a different block type for each visualization type because they each expect different arguments, and so while I would agree that it’s pretty annoying to write each one out separately, it does allow Terraform to do its usual validation work to see that the contents of each content block conform to that resource type schema in particular.

A quirk of the design I described above is that without any additional validation it won’t catch unsupported visualization types, and so would just silently ignore them whereas I expect the previous design would’ve caught that as a provider-side validation error. To recover that validation behavior I might add a validation block to the definition of the widgets input variable:

variable "widgets" {
  type = list(object({
    visualization = string
    # ...
  }))

  validation {
    condition = alltrue([
      for w in var.widgets : contains(["bar", "billboard", "bullet", "markdown"], w.visualization)
    ])
    error_message = "Not all widgets have supported visualization types."
  }
}

This is not an ideal error message because it requires the reader of the message to go figure out which of the widgets is wrong and in what way by reading the calling code, but I’m sharing this only in the interests of it being better than the alternative of an unsupported type just being ignored entirely, with no direct feedback whatsoever.

I’m not very familiar with New Relic so I’ve answered here primarily in general Terraform terms, but I hope you’ll be able to use this as a basis for your new module version.

[aaa]

This is all very interesting and very helpful, thank you @apparentlymart!

Ah, so with this code in the child module inside the newrelic_one_dashboard resource,

  dynamic "widget_billboard" {
    for_each = try(local.widgets_by_viz.billboard, [])
    content {
      # ...
    }
  }

  dynamic "widget_markdown" {
    for_each = try(local.widgets_by_viz.markdown, [])
    content {
      # ...
    }
  }

If I want my child module to be able to support all the new widget types for the newrelic_one_dashboard resource, then in that child module I will need a dynamic "" { } block for each kind of new widget block - meaning widget_bar, widget_billboard, widget_bullet , etc - and which from what I counted is 12 unique kinds of widgets in that newrelic_one_dashboard doc. So basically 12 dynamic { } blocks are needed. Is that understanding correct?

If that is correct, I also presume that if for some reason a person wanted to created create more than one particular type of widget - say they wanted two widget_bullets in their dashboard - they can do that with the above type of setup as well.

[apparentlymart]

Yes, you would need to write out a separate dynamic block to explain how to generate blocks of each of the supported types. This is an unfortunate consequence of the new design of this resource type, although I imagine it comes in return for the benefit of allowing them to vary the schema for each one, whereas before they presumably had to just make widget the superset of all of the different possible arguments and then do some later validation inside the provider code itself to catch the impossible combinations… a tricky design tradeoff, for sure.

The reason I used try(local.widgets_by_viz.type, []) here rather than just local.widgets_by_viz.type is to deal with different combinations of types. The for grouping mode ... will only create entries in the map for visualization types that appear at least once in the input, so the try here is arranging to fall back to an empty list if one doesn’t appear at all, which then means that dynamically Terraform will generate zero blocks of that type. That means that in the case where you have one billboard widget and one markdown widget, for example, Terraform would generate one widget_billboard block, one widget_markdown block, and zero widget_bullet or any other type of block you’ve defined in this way, and thus get the effect of allowing any combination of the supported widget types.

[aaa]

Makes sense, thanks again @apparentlymart and hope to get this implemented!

[aaa]

@apparentlymart

locals {
  widgets_by_viz = {
    for w in var.widgets : w.visualization => w...
  }
}

@apparentlymart, just for clarification. the above code was meant to go in the child module that has the newrelic_one_dashboard resource of course, having it’s own declared widgets variable (I do have that variable in the child module already of course)? Meaning, the child module takes in the widgets from the root module via that line in my root module.

terraform/main.tf

module "newrelic_dashboard" {
    ...
     widgets       = local.dynamic_widgets

and then each of those 12 dynamic widget blocks you mentioned - which would also be in the child module - such as dynamic "widget_markdown" { .. } would take those in.

Basically, all the code you shared would be in the child module. If so, I think I can take it from there but wanted to be 100% sure.

Additional Question

this has me thinking that the child module main.tf might get a little bloated with 12 dynamic { } blocks, one for each visualization type. Would it be possible to move all those dynamic blocks into a separate file, such as widget_dynamics.tf or so, and then refer to that file / its contents in the newrelic_one_dashboard { } resource in main.tf? I was thinking perhaps some type of usage of the file() function, but I’m not sure that that would work within a resource, in this case inside the page { } block of the newrelic_one_dashboard resource. To have it handy, the basic structure of the newrelic_one_dashboard is:

resource "newrelic_one_dashboard" "exampledash" {
  name = "New Relic Terraform Example"

  page {
    name = "New Relic Terraform Example"

    widget_billboard {
      title = "Requests per minute"
      row = 1
      column = 1

      nrql_query {
        query       = "FROM Transaction SELECT rate(count(*), 1 minute)"
      }
    }

    widget_bar {
...

Of course, this is all happening in that child module containing that newrelic_one_dashboard resource, in main.tf

[apparentlymart]

Hi @aaa,

Yes, my understanding of your requirement here was to change the behavior of your module without changing its public interface, so everything I’ve suggested here has been under the assumption that you’re modifying the internals of the module and not anything outside of it. Any existing callers of this module will presumably still see Terraform plan to destroy the old resource type and create the new one, but it’ll happen without them needing to change their module call at all. I hope that wasn’t a misunderstanding of your intentions!

I can’t think of any way to factor out the dynamic blocks from inside the newrelic_one_dashboard block. Terraform typically wants the entirety of the definition of an object all in one place, so it’s clear where it’s all defined, but indeed in this case it does mean you’ll end up with a pretty long resource "newrelic_one_dashboard" block. But at least this resource is already in a module and so it’s factored out of the configurations that will be calling it, so the actual concrete definitions of the widgets should presumably still be relatively concise even though the mapping from that definition to the resource configuration is not, and you only need to write that mapping out once to be called many times.

[aaa]

@apparentlymart

I hope that wasn’t a misunderstanding of your intentions!

No, not at all, spot on on what I was trying to do. I may need to modify the calling code only in terms of specific widget attributes in case those have changed - I just haven’t looked at that as of late, and don’t expect that to be too much.

Hmm, if I can’t take out the widget blocks to a separate file, I could potentially take out the newrelic_one_dashboard entirely from this module’s main.tf and perhaps put that resource into say a file named dashboard.tf. But the whole point of this module was just the dashboard, so that really might not be necessary and bloat isn’t a real concern, since the dashboard is the whole point/substance of the module. Thanks again @apparentlymart !

[aaa]

@apparentlymart

The best way I can think of to work with this new resource schema design is to first preprocess the list of widgets into a map of lists grouped by type, like this:

locals {
  widgets_by_viz = {
    for w in var.widgets : w.visualization => w...
  }
}

This is a for expression with the ... grouping mode enabled, and so the result will be a map of lists where the map keys are all of the distinct visualization types in the input.

Hello again @apparentlymart, I just wanted to be sure I understood and also am seeking clarification, specifically on this part:

map of lists where the map keys are all of the distinct visualization types in the input.

Is it a map (hash) of lists (an array), or is it actually a map of maps? Perhaps I just needed to be sure by drilling further, and it is a map of lists, each list containing maps. I was trying to related to the example of the grouping doc you linked to, which has this:

{
  "admin": [
    "ps",
  ],
  "maintainer": [
    "am",
    "jb",
    "kl",
    "ma",
  ],
  "viewer": [
    "st",
    "zq",
  ],
}

Wouldn’t my data in particular be something like this? So perhaps more specifically it’s a map of lists, each list containing maps?

{
  "line": [
     {
      title         = "Line Widget"
      visualization = "line"
      entity_ids    = [data.newrelic_entity.my_application.application_id, ]
      nrql          = "SELECT rate(count(*), 1 minute) FROM Transaction WHERE appName = '${var.nr_entity_name}'"
      row           = 1
      column        = 1
    },
  ],
  "billboard": [
    { title         = "First Billboard"
      visualization = "billboard"
      entity_ids    = [data.newrelic_entity.my_application.application_id, ]
      nrql          = "SELECT rate(count(*), 1 minute) FROM Transaction WHERE appName = '${var.nr_entity_name}'"
      row           = 1
      column        = 1
    },
    { title         = "Second Billboard"
      visualization = "billboard"
      entity_ids    = [data.newrelic_entity.my_application.application_id, ]
      nrql          = "SELECT rate(count(*), 1 minute) FROM Transaction WHERE appName = '${var.nr_entity_name}'"
      row           = 1
      column        = 1
    },
  ],
}

Just wanted to be sure!