Optional Computed Block handling in Plugin Framework

maastha · July 21, 2023, 9:13am

We are migrating from SDKv2 to Plugin Framework. In SDKv2 implementation of a resource we have a schema.TypeSet which is Optional and Computed.
This means the value of this set could be different than what the user might have in the config. For example, if the user did not provide this set in the configuration, the API could possibly return the set with one default element object, or another case can be, if user specified 2 elements the API can possibly return 3 elements during.

In the migrated resource, we specified this Set as a NestedBlock:

"api_keys": schema.SetNestedBlock{
				NestedObject: schema.NestedBlockObject{
					Attributes: map[string]schema.Attribute{
						"api_key_id": schema.StringAttribute{
							Required: true,
						},
						"role_names": schema.SetAttribute{
							Required:    true,
							ElementType: types.StringType,
						},
					},
				},
				PlanModifiers: []planmodifier.Set{
				},
			},

Do I need to mark the block as Computed and Optional? If so, how can I do that? My understanding is blocks don’t support those attributes.
In the Create handler of the resource, how do I handle the different incoming values/elements of this resource than the configuration to avoid this error:

│ Error: Provider produced inconsistent result after apply
│ When applying changes to ... provider ... produced an unexpected new value: .api_keys: block set length changed from 2 to 3.

How do I handle this Set during Read handler such that every time user runs terraform plan it doesn’t mark this attribute as changed?

I believe there is a way to do this using PlanModifiers but I’m not sure how.

Please let us know how this should be approached, would really appreciate an example.
Thank you!

bendbennett · July 24, 2023, 4:56pm

Hi @maastha

In terms of implementation within the Framework, I believe the choices are either to use a SetAttribute if it you want to avoid breaking changes or you could switch to using SetNestedAttribute if breaking changes are acceptable. Blocks With Computed Fields in the migration guide contains some information on this area.

The schema would look something like the following when using SetAttribute:

			"api_keys": schema.SetAttribute{
				Computed: true,
				Optional: true,
				ElementType: types.ObjectType{
					AttrTypes: map[string]attr.Type{
						"api_key_id": types.StringType,
						"role_names": types.SetType{
							ElemType: types.StringType,
						},
					},
				},
			},
			/* ... */

The schema would look something like the following when using SetNestedAttribute:

			"api_keys": schema.SetNestedAttribute{
				Computed: true,
				Optional: true,
				NestedObject: schema.NestedAttributeObject{
					Attributes: map[string]schema.Attribute{
						"api_key_id": schema.StringAttribute{
							Required: true,
						},
						"role_names": schema.SetAttribute{
							ElementType: types.StringType,
							Required:    true,
						},
					},
				},
			},
			/* ... */

If you are verifying that the configuration is unknown then you should be able to run something like the following in the Create function:

	if data.ApiKeys.IsUnknown() {
		data.ApiKeys = types.SetValueMust(
			types.ObjectType{
				AttrTypes: map[string]attr.Type{
					"api_key_id": types.StringType,
					"role_names": types.SetType{
						ElemType: types.StringType,
					},
				},
			}, []attr.Value{
				types.ObjectValueMust(
					map[string]attr.Type{
						"api_key_id": types.StringType,
						"role_names": types.SetType{
							ElemType: types.StringType,
						},
					},
					map[string]attr.Value{
						"api_key_id": types.StringValue("api_key_id_one"),
						"role_names": types.SetValueMust(
							types.StringType,
							[]attr.Value{
								types.StringValue("role_name_one"),
							},
						),
					},
				),
			},
		)
	}

If the value of api_keys changes during the Read operation then the following would result in an update to the state without marking the attribute as changed:

	data.ApiKeys = types.SetValueMust(
		types.ObjectType{
			AttrTypes: map[string]attr.Type{
				"api_key_id": types.StringType,
				"role_names": types.SetType{
					ElemType: types.StringType,
				},
			},
		}, []attr.Value{
			types.ObjectValueMust(
				map[string]attr.Type{
					"api_key_id": types.StringType,
					"role_names": types.SetType{
						ElemType: types.StringType,
					},
				},
				map[string]attr.Value{
					"api_key_id": types.StringValue("api_key_id_one"),
					"role_names": types.SetValueMust(
						types.StringType,
						[]attr.Value{
							types.StringValue("role_name_one"),
						},
					),
				},
			),
			types.ObjectValueMust(
				map[string]attr.Type{
					"api_key_id": types.StringType,
					"role_names": types.SetType{
						ElemType: types.StringType,
					},
				},
				map[string]attr.Value{
					"api_key_id": types.StringValue("api_key_id_two"),
					"role_names": types.SetValueMust(
						types.StringType,
						[]attr.Value{
							types.StringValue("role_name_two"),
						},
					),
				},
			),
		},
	)

	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)

maastha · July 24, 2023, 5:35pm

@bendbennett My understanding was that we should be using Blocks (hence, schema.SetNestedBlock) for migrating collections (also, using protocol5to6) to avoid any breaking changes when migrating to the framework because using Attributes (SetAttribute or SetNestedAttribute) would require users to update their configs to use new syntax with “=” sign. Is this not correct? If correct, then moving to attributes would be a breaking change, hence, Can you advise how to achieve our use-case of Computed and optional with Blocks?

Blocks with Computed Fields shares guidance on how to migrate “computed-only” blocks. But in our case the concerned field “api_keys” is optional too, meaning user can provide this value in their config as well but final value will be determined as returned by the API in CREATE response (and thereafter, every subsequent READ/GET response).

Please guide us on how to correctly migrate this scenario without any breaking changes. As mentioned, in our SDKv2 implementation, this collection is a schema.TypeSet (Computed and Optional)

bendbennett · July 25, 2023, 1:06pm

Hi @maastha,

You are correct that switching to an attribute would represent a breaking change. You are also correct that the link I supplied for Blocks with Computed Fields is for “computed-only ” blocks, my apologies.

If you are using a SetNestedBlock as you have illustrated, then the block is already effectively optional. If api_keys is not defined in the Terraform configuration that is supplied then an empty set will be written into state.

However, if you need to be able to handle instances in which api_keys is supplied in the Terraform configuration and you have values returned from your API which you need to store then you will need to add an additional computed attribute or attributes to hold this data as Terraform will not allow modification of data that was supplied through the configuration. Modification of this sort is what gives rise to the error that you reported (e.g., produced an unexpected new value....).

Alternatively, if you do not need to store the additional information that is returned by your API call you can filter it out so that the only data that is written to state is the data supplied in the Terraform configuration.

In short:

You do not need to mark the block as Optional, the block is already effectively optional. If you have data that is “computed” (i.e., returned from an API call) that needs to be stored then you will need to alter your schema to add a computed attribute or attributes that will be used to hold this data. If you do not need to persist data that is returned from an API call you can filter it out to make sure that it doesn’t mutate the value in api_keys.
You can avoid errors such as unexpected new value: ... by making sure that you do not attempt to mutate any value for api_keys that has been set in the configuration. Again you will need to use an additional computed attribute or attributes for data returned from your API call.
You can avoid terraform plan from marking the attribute as changed by filtering out the data returned from the API call.

Note that addition of a computed attribute or attributes represents a breaking change but this may be unavoidable depending upon whether you need to persist the data returned from your API call or not.

sikaleel87 · January 19, 2024, 11:37am

Hi @bendbennett - This is really awesome. i tried this code and it solved my need perfectly.

one doubt is it possible to process this set value details in loop for multiple objects?

for example, my data will look like below

{
"subusers": [
        {
            "user_id": 12345678,
            "username": "chida.test"
        },
        {
            "user_id": 87654321,
            "username": "sk.dev"
        }
    ]
//other values to
}

sikaleel87 · January 19, 2024, 12:37pm

Thank you @bendbennett i solved it by myself as mentioned here.