Set/list attribute migration from SDKv2 to framework

yec-akamai · July 25, 2023, 2:24pm

We have a couple of set/list attributes in the resource schema to migrate from SDKv2 to terraform-plugin-framework. Some of them is tricky to work. One is a computed, optional SetAttribute, with some required attributes and some attributes need default. For example:

"sdkv2_set_attribute": {
	Type:        schema.TypeSet,
	Description: "A set",
	Optional:    true,
	Computed:    true,
	Elem:        setElement(),
}

func setElement() *schema.Resource {
	return &schema.Resource{
		Schema: map[string]*schema.Schema{
			"attr_1": {
				Type:        schema.TypeInt,
				Required:    true,
				Description: "attr1",
			},
			"attr_2": {
				Type:        schema.TypeBool,
				Description: "attr2",
				Optional:    true,
				Default:     false,
			},
		}
	}
}

We’ve tried some options but neither of them works well so far:

Nested attribute: we can’t use it because we must support protocol 5
Nested block: it can include these attributes settings, however, the block must be inputed in the config and the required attributes must be set in the case. Because the list/set is optional in sdkv2, user usually doesn’t have it in the config. Without the block in the config, this error is raised:

        Error: Provider produced inconsistent result after apply
        
        When applying changes to my_resource.test, provider
        "provider[\"registry.terraform.io/hashicorp/linode\"]" produced an unexpected
        new value: .my_set_block: actual set element
        cty.ObjectVal(map[string]cty.Value{"id":some_id,
        "label":some_label) does not correlate with any element in
        plan.
        
        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
        Error: Provider produced inconsistent result after apply
        
        When applying changes to my_resource.test, provider
        "provider[\"registry.terraform.io/hashicorp/linode\"]" produced an unexpected
        new value: .my_set_block: block set length changed from 0 to 1.

List/set attribute + object type: it can’t have complicated setting, i.e. defaults, required, etc. I tried to set default in ModifyPlan() but doesn’t work:

Error: Provider produced invalid plan
        
        Provider "registry.terraform.io/hashicorp/linode" planned an invalid value
        for my_resource.test.my_list: planned value
        cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"my_attr":cty.StringVal("my_default_string")...}

does not match config value
cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"my_attr":cty.NullVal(cty.String)...}

Can we have any suggestion on what’s the approach we should go with and how may we fix the issues discovered? Thank you in advance.

bendbennett · July 26, 2023, 7:37am

Hi @yec-akamai

It sounds like you’ve already looked through the documentation for migrating blocks from SDKv2 to the Framework.

In terms of using a nested block, then if you need to make supplying of Terraform configuration for the block optional and you have a Required attribute on the block then you won’t be able to use a SingleNestedBlock, for instance:

		Blocks: map[string]schema.Block{
			"b": schema.SingleNestedBlock{
				Attributes: map[string]schema.Attribute{
					"attr_1": schema.Int64Attribute{
						Required: true,
					},
					"attr_2": schema.BoolAttribute{
						Optional:    true,
					},
				},
			},
		},

This is because omission of configuration for the block will give rise to the following error:

│ Must set a configuration value for the b.attr_1 attribute as the provider has marked it as required.

However, you could use SetNestedBlock and add a validator, for example:

		Blocks: map[string]schema.Block{
			"b": schema.SetNestedBlock{
				NestedObject: schema.NestedBlockObject{
					Attributes: map[string]schema.Attribute{
						"attr_1": schema.Int64Attribute{
							Required: true,
						},
						"attr_2": schema.BoolAttribute{
							Optional:    true,
						},
					},
				},
				Validators: []validator.Set{
					setvalidator.SizeAtMost(1),
				},
			},
		},

The error that you describe (in points 2. and 3.) is a consequence of the value for the block being altered from the value defined in the Terraform configuration. This error is expected as Terraform will not allow modification of data that was supplied through the configuration, including cases in which no configuration was supplied as the expectation is then that the value will be null.

If there are “computed” values that are being obtained from an API call, for instance, and these need to be persisted then you will need to amend your schema to include an additional computed attribute or attributes. Note that addition of a computed attribute or attributes represents a breaking change but this may be unavoidable depending upon whether you need to persist the data returned from your API call or not. There is discussion around a similar use-case in Optional Computed Block handling in Plugin Framework

yec-akamai · July 26, 2023, 2:36pm

Hi @bendbennett

Thank you so much for the detailed explanation! It’s very helpful to understand the issue and how the config works. I also read your response in the other discussion, and we do have a very similar issue.

I think as you suggested, have an additional computed attribute to obtain values from API call is probably the solution in this case, though it’s a breaking change indeed. Thanks again!

zliang-akamai · July 28, 2023, 8:09pm

Hi @bendbennett

Thank you for the answer! Do you think if we can have a seamless migration experience with Nested Attributes and protocol 6 for this type of issue in the future? And may I know if there is any public plan for putting TF<1.0 to be EOL so we can push to protocol 6?

bendbennett · August 1, 2023, 7:22am

Hi @zliang-akamai

There is a HashiCorp Support post relating to 0.12.X Support Period and End-of-Life (EOL). It’s generally encouraged to use the most recent version of Terraform that you can.

In relation to your question about “a seamless migration experience with Nested Attributes and protocol 6”, it depends on your use-case. Please feel free to open a new issue if you have a specific question about migrating from SDkv2 to the plugin Framework or switching from protocol version 5 to protocol version 6.

zliang-akamai · September 14, 2023, 10:34pm

Hi @bendbennett,

Thanks for the response!

Like this case brought by @yec-akamai, this is an important feature supported by SDKv2 but went missing in the Framework with protocol 5.

If TF team doesn’t plan to put TF<1.0 to be EOL and to enforce protocol 6 anytime soon, can we have this feature (computed block with a fully functional schema) in framework with protocol 5? It wouldn’t be easy for plugin developers to implement either a workaround or a resource in SDKv2 and then later migrate it to the framework. As it’s supported by SDKv2, I don’t think the protocol 5 is really a blocker.

bendbennett · September 15, 2023, 7:17am

Hi @zliang-akamai ,

Can you provide a little more detail on what you have in mind when you say “computed block with a fully functional schema”? If you can provide an example SDKv2 schema and the equivalent that you would like to see in a Framework schema (pseudo-code if necessary) that illustrates what you are thinking about that would be really helpful.

In general terms, the migration path outlined above represents the current suggested approach for migrating from SDKv2 to the Framework for the case described.