Plugin Framework - Double Nested?

Terraform Providers Plugin Development
1

I’m working a provider and have an API that returns data back to me with some twice (or even further) nested attributes. Wondering the best way to handle this in the schema. Been running into issues trying to accomplish this.

The data looks like:

{
  "state": "paused",
  "instructions": {
    "actions": {"name": "my action", "config": {"port": 1234, "host": "my.com"}},
   "filters": [{"name": "filter1", "config": {"key": "value1"}, "on_run": true}, {"name": "filter2", "config": {"key": "value2"}, "on_run": true}]
    "groups": ["group1", "group2", "group3"]
  }
}
2

There are examples of how I like to do it here.

Long story short, each nested layer is an object with both a DataSourceAttributes() and ResourceAttributes() method.

Sometimes they also have read-only versions of those methods, if the same object shows up in both writable and read-only Resource or Datasource contexts.

3

Whenever I try doing similar I always end up with Value Conversion Errors when I attempt to apply the values of the instructions — a conflict between the model used and the object type it expects but I’m not sure how to convert appropriately since the struct has to tie back to the defining structure of its contents.

Details of Value Conversion error:

│ Received null value, however the target type cannot handle null values. Use the corresponding `types` package type, a pointer type or a custom type that handles null values.
│ 
│ Path: instructions
│ Target Type: insightcloudsec.instructionsModel
│ Suggested `types` Type: basetypes.ObjectValue
│ Suggested Pointer Type: *insightcloudsec.instructionsModel
╵
4

If you poke around in the code I linked, you’ll find zero (I hope!) instances of a struct which has elements of a native go type with a tfsdk tag applied.

Never this:

type thingy struct {
  SomeString      string   `tfsdk:"some_string"`
  SomeInt         int64    `tfsdk:"some_int"`
  SomeStringList  []string `tfsdk:"some_string_list"`
}

Always this:

type thingy struct {
  SomeString.    types.String `tfsdk:"some_string"`
  SomeInt        types.Int64  `tfsdk:"some_int"`
  SomeStringList types.List   `tfsdk:"some_string_list"`
}

That has resolved 99% of the problems I’ve had with those errors (at a cost of doing lots of back-and-forth conversion when I want to actually use the values).

5

I always use the types - except for when I have nested structs like this where I reference another struct using the types. In other words, all my attributes of the strut are using the types.String etc except the instruction attribute which is using instructionsModel - another struct that is made up of attributes using the types.

6

Use a types.Object there instead?

This is almost certainly the issue.

You could use a *SomeStruct, because pointers can be nil (null)…

…But they can’t be unknown, so a types.Object is still safer (depending on where you plan to go with various capabilities).

There was I time that I tried to pick and choose where I’d use native types vs. something from the framework types library and I came to regret it.

7

No go. I still get a Value Conversion Error using the types.Object. Here’s what I originally have for the structs:

type botDataSourceModel struct {
	ID           types.String         `tfsdk:"resource_id"`
	Name         types.String         `tfsdk:"name"`
	Description  types.String         `tfsdk:"description"`
	Owner        types.String         `tfsdk:"owner"`
	OwnerName    types.String         `tfsdk:"owner_name"`
	Scope        types.Set            `tfsdk:"scope"`
	Severity     types.String         `tfsdk:"severity"`
	State        types.String         `tfsdk:"state"`
	Instructions botInstructionsModel `tfsdk:"instructions"`
}

type botInstructionsModel struct {
	Actions             []botActionsModel `tfsdk:"actions"`
	Badges              []badgesModel     `tfdsk:"badges"`
	ExclusionBadges     []badgesModel     `tfsdk:"exclusion_badges"`
	Filters             []filtersModel    `tfsdk:"filters"`
	Groups              types.Set         `tfsdk:"groups"`
	Hookpoints          types.Set         `tfsdk:"hookpoints"`
	ResourceTypes       types.Set         `tfsdk:"resource_types"`
	Schedule            types.String      `tfsdk:"schedule"`
	ScheduleDescription types.String      `tfsdk:"schedule_description"`
}

type botActionsModel struct {
	Name            types.String `tfsdk:"name"`
	Config          types.Map    `tfsdk:"config"`
	RunWhenResultIs types.Bool   `tfsdk:"run_when_result_is"`
}

And here’s the schema definitions:

func (d *botDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
	resp.Schema = schema.Schema{
		Attributes: map[string]schema.Attribute{
			"resource_id": schema.StringAttribute{
				Required:            true,
				Description:         "Bot ID",
				MarkdownDescription: "The bot ID associated with the bot to return.  For example: divvybot:1:123.",
			},
			"name": schema.StringAttribute{
				Computed:            true,
				Description:         "Bot Name",
				MarkdownDescription: "The name of the bot.",
			},
			"description": schema.StringAttribute{
				Computed:            true,
				Description:         "Description of the bot",
				MarkdownDescription: "The description given to the bot.",
			},
			"severity": schema.StringAttribute{
				Computed:            true,
				Description:         "Severity of the bot",
				MarkdownDescription: "The severity given to the bot.",
				Validators: []validator.String{
					stringvalidator.OneOfCaseInsensitive("low", "medium", "high"),
				},
			},
			"owner": schema.StringAttribute{
				Computed:            true,
				Description:         "Bot Owner",
				MarkdownDescription: "The bot owner's account.",
			},
			"owner_name": schema.StringAttribute{
				Computed:            true,
				Description:         "Bot Owner's Name",
				MarkdownDescription: "The bot owner's name.",
			},
			"scope": schema.SetAttribute{
				Computed:            true,
				Description:         "Scope for the bot",
				MarkdownDescription: "Scopes of the given bot.",
				ElementType:         types.StringType,
			},
			"state": schema.StringAttribute{
				Computed:            true,
				Description:         "Current State of the bot",
				MarkdownDescription: "The current state given to the bot.",
				Validators: []validator.String{
					stringvalidator.OneOfCaseInsensitive("paused", "running"),
				},
			},
			"instructions": schema.SingleNestedAttribute{
				Computed:            true,
				Description:         "Bot Instructions",
				MarkdownDescription: "The bot instructions that define how the bot is to operate and on what.",
				Attributes: map[string]schema.Attribute{
					"actions": schema.ListNestedAttribute{
						Computed:            true,
						Description:         "Bot actions",
						MarkdownDescription: "The actions the bot is to take when run.",
						NestedObject: schema.NestedAttributeObject{
							Attributes: map[string]schema.Attribute{
								"name": schema.StringAttribute{
									Computed:            true,
									Description:         "Action's Name",
									MarkdownDescription: "The name of the bot action.",
								},
								"config": schema.MapAttribute{
									Computed:            true,
									Description:         "Configs for the filter",
									MarkdownDescription: "Configuration settings for the given filter.",
									ElementType:         types.StringType,
								},
								"run_when_result_is": schema.BoolAttribute{
									Computed:            true,
									Description:         "Run when the result is",
									MarkdownDescription: "Run the action when result is true or false.",
								},
							},
						},
					},
					"badges": schema.ListNestedAttribute{
						Computed:            true,
						Description:         "Badges applied to the bot",
						MarkdownDescription: "The badges applied to the bot.",
						NestedObject: schema.NestedAttributeObject{
							Attributes: map[string]schema.Attribute{
								"key": schema.StringAttribute{
									Computed: true,
								},
								"value": schema.StringAttribute{
									Computed: true,
								},
							},
						},
					},
					"exclusion_badges": schema.ListNestedAttribute{
						Computed:            true,
						Description:         "Excluded badges for the bot",
						MarkdownDescription: "The badges that are excluded from this bot.",
						NestedObject: schema.NestedAttributeObject{
							Attributes: map[string]schema.Attribute{
								"key": schema.StringAttribute{
									Computed: true,
								},
								"value": schema.StringAttribute{
									Computed: true,
								},
							},
						},
					},
					"filters": schema.ListNestedAttribute{
						Computed:            true,
						Description:         "Filters applied to the bot.",
						MarkdownDescription: "The filters used to query resources in the bot.",
						NestedObject: schema.NestedAttributeObject{
							Attributes: map[string]schema.Attribute{
								"name": schema.StringAttribute{
									Computed:            true,
									Description:         "Filter name",
									MarkdownDescription: "The name of the filter",
								},
								"config": schema.MapAttribute{
									Computed:            true,
									Description:         "Configs for the filter",
									MarkdownDescription: "Configuration settings for the given filter.",
									ElementType:         types.StringType,
								},
							},
						},
					},
					"groups": schema.SetAttribute{
						Computed:            true,
						Description:         "Groups to apply the bot",
						MarkdownDescription: "Groups to apply the bot.",
						ElementType:         types.StringType,
					},
					"hookpoints": schema.SetAttribute{
						Computed:            true,
						Description:         "Hookpoints for the bot",
						MarkdownDescription: "Hookpoints for the bot.",
						ElementType:         types.StringType,
					},
					"resource_types": schema.SetAttribute{
						Computed:            true,
						Description:         "Resource types used for the bot",
						MarkdownDescription: "Resource types used for the bot.",
						ElementType:         types.StringType,
					},
					"schedule": schema.StringAttribute{
						Computed:            true,
						Description:         "Bot Schedule",
						MarkdownDescription: "A string represnting a json object for the bot's schedule.",
					},
					"schedule_description": schema.StringAttribute{
						Computed:            true,
						Description:         "Bot Schedule Description",
						MarkdownDescription: "A string represnting a json object for the bot's schedule description.",
					},
				},
			},
		},
	}
}
8

This could be:

Instructions types.Object `tfsdk:"instructions"`

…so that it can handle “null” and “unknown” conditions.

Same for:

Actions             []botActionsModel `tfsdk:"actions"`
Badges              []badgesModel     `tfdsk:"badges"`
ExclusionBadges     []badgesModel     `tfsdk:"exclusion_badges"`
Filters             []filtersModel    `tfsdk:"filters"`

They could be types.Set or types.List.

1 Like
9

Instructions contains all of those fields – Actions, Badges, ExclusionBadges, etc. So how would they relate if I made instructions types.Object?

And some of those have multiple fields within them. Actions for example contains a Name, Config and Boolean value for each entry in the List with its own.

10

You’ll still need to keep the botInstructionsModel struct hanging around. You then unpack the types.Object into an instance of botInstructionsModel using botDataSourceModel.Instructions.As()

1 Like
11

How do I go the other direction – from the model to the Object? The values are all computed. Only option I see is botDataSourceModel.Instruction.ToObjectValue() but that only wants context.

12

Use types.ObjectValueFrom(), types.ListValueFrom(), etc…

1 Like
13

Thank you - this makes way more sense now.

14

The need to describe the type (map[string]attr.Type) in the *ValueFrom() methods, combined with lots of re-use of nested objects is how all of my nested “model” structs came to have their own Attributes() and AttrTypes() methods.