Set/list attribute migration from SDKv2 to framework

Terraform Providers Plugin Development
1

We have a couple of set/list attributes in the resource schema to migrate from SDKv2 to terraform-plugin-framework. Some of them is tricky to work. One is a computed, optional SetAttribute, with some required attributes and some attributes need default. For example:

"sdkv2_set_attribute": {
	Type:        schema.TypeSet,
	Description: "A set",
	Optional:    true,
	Computed:    true,
	Elem:        setElement(),
}

func setElement() *schema.Resource {
	return &schema.Resource{
		Schema: map[string]*schema.Schema{
			"attr_1": {
				Type:        schema.TypeInt,
				Required:    true,
				Description: "attr1",
			},
			"attr_2": {
				Type:        schema.TypeBool,
				Description: "attr2",
				Optional:    true,
				Default:     false,
			},
		}
	}
}

We’ve tried some options but neither of them works well so far:

  1. Nested attribute: we can’t use it because we must support protocol 5

  2. Nested block: it can include these attributes settings, however, the block must be inputed in the config and the required attributes must be set in the case. Because the list/set is optional in sdkv2, user usually doesn’t have it in the config. Without the block in the config, this error is raised:

        Error: Provider produced inconsistent result after apply
        
        When applying changes to my_resource.test, provider
        "provider[\"registry.terraform.io/hashicorp/linode\"]" produced an unexpected
        new value: .my_set_block: actual set element
        cty.ObjectVal(map[string]cty.Value{"id":some_id,
        "label":some_label) does not correlate with any element in
        plan.
        
        This is a bug in the provider, which should be reported in the provider's own
        issue tracker.
        Error: Provider produced inconsistent result after apply
        
        When applying changes to my_resource.test, provider
        "provider[\"registry.terraform.io/hashicorp/linode\"]" produced an unexpected
        new value: .my_set_block: block set length changed from 0 to 1.
  1. List/set attribute + object type: it can’t have complicated setting, i.e. defaults, required, etc. I tried to set default in ModifyPlan() but doesn’t work:
Error: Provider produced invalid plan
        
        Provider "registry.terraform.io/hashicorp/linode" planned an invalid value
        for my_resource.test.my_list: planned value
        cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"my_attr":cty.StringVal("my_default_string")...}

does not match config value
cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"my_attr":cty.NullVal(cty.String)...}

Can we have any suggestion on what’s the approach we should go with and how may we fix the issues discovered? Thank you in advance.

2

Hi @yec-akamai :wave:

It sounds like you’ve already looked through the documentation for migrating blocks from SDKv2 to the Framework.

In terms of using a nested block, then if you need to make supplying of Terraform configuration for the block optional and you have a Required attribute on the block then you won’t be able to use a SingleNestedBlock, for instance:

		Blocks: map[string]schema.Block{
			"b": schema.SingleNestedBlock{
				Attributes: map[string]schema.Attribute{
					"attr_1": schema.Int64Attribute{
						Required: true,
					},
					"attr_2": schema.BoolAttribute{
						Optional:    true,
					},
				},
			},
		},

This is because omission of configuration for the block will give rise to the following error:

│ Must set a configuration value for the b.attr_1 attribute as the provider has marked it as required.

However, you could use SetNestedBlock and add a validator, for example:

		Blocks: map[string]schema.Block{
			"b": schema.SetNestedBlock{
				NestedObject: schema.NestedBlockObject{
					Attributes: map[string]schema.Attribute{
						"attr_1": schema.Int64Attribute{
							Required: true,
						},
						"attr_2": schema.BoolAttribute{
							Optional:    true,
						},
					},
				},
				Validators: []validator.Set{
					setvalidator.SizeAtMost(1),
				},
			},
		},

The error that you describe (in points 2. and 3.) is a consequence of the value for the block being altered from the value defined in the Terraform configuration. This error is expected as Terraform will not allow modification of data that was supplied through the configuration, including cases in which no configuration was supplied as the expectation is then that the value will be null.

If there are “computed” values that are being obtained from an API call, for instance, and these need to be persisted then you will need to amend your schema to include an additional computed attribute or attributes. Note that addition of a computed attribute or attributes represents a breaking change but this may be unavoidable depending upon whether you need to persist the data returned from your API call or not. There is discussion around a similar use-case in Optional Computed Block handling in Plugin Framework

1 Like
3

Hi @bendbennett :wave:

Thank you so much for the detailed explanation! It’s very helpful to understand the issue and how the config works. I also read your response in the other discussion, and we do have a very similar issue.

I think as you suggested, have an additional computed attribute to obtain values from API call is probably the solution in this case, though it’s a breaking change indeed. Thanks again!

4

Hi @bendbennett

Thank you for the answer! Do you think if we can have a seamless migration experience with Nested Attributes and protocol 6 for this type of issue in the future? And may I know if there is any public plan for putting TF<1.0 to be EOL so we can push to protocol 6?

5

Hi @zliang-akamai :wave:

There is a HashiCorp Support post relating to 0.12.X Support Period and End-of-Life (EOL). It’s generally encouraged to use the most recent version of Terraform that you can.

In relation to your question about “a seamless migration experience with Nested Attributes and protocol 6”, it depends on your use-case. Please feel free to open a new issue if you have a specific question about migrating from SDkv2 to the plugin Framework or switching from protocol version 5 to protocol version 6.

6

Hi @bendbennett,

Thanks for the response!

Like this case brought by @yec-akamai, this is an important feature supported by SDKv2 but went missing in the Framework with protocol 5.

If TF team doesn’t plan to put TF<1.0 to be EOL and to enforce protocol 6 anytime soon, can we have this feature (computed block with a fully functional schema) in framework with protocol 5? It wouldn’t be easy for plugin developers to implement either a workaround or a resource in SDKv2 and then later migrate it to the framework. As it’s supported by SDKv2, I don’t think the protocol 5 is really a blocker.

7

Hi @zliang-akamai :wave:,

Can you provide a little more detail on what you have in mind when you say “computed block with a fully functional schema”? If you can provide an example SDKv2 schema and the equivalent that you would like to see in a Framework schema (pseudo-code if necessary) that illustrates what you are thinking about that would be really helpful.

In general terms, the migration path outlined above represents the current suggested approach for migrating from SDKv2 to the Framework for the case described.

8

Hi @bendbennett,
Sorry about the late reply. We recently re-iterate the framework migration work.

I think the current Framework offering can’t fulfill what SDKv2 does in this example. May I have some suggestions from TF Framework team? Do you guys have a plan to implement something similar to what’s in SDKv2? Thank you!

I built this SDKv2 example that’s hard to migrate Framework:

With an example of naive and failing framework migration code:

9

Hi @zliang-akamai :wave:

Sorry you ran into trouble here.

Thank you for sharing the schema.

Can you provide some information on the issues you are encountering?
Are you seeing errors, breaking changes to the configuration etc?

If you could also supply the Terraform configuration you are using that would be helpful.

10

Hi @bendbennett,

Basically, the block can no longer be computed (being modified to a value different than the user configured value during Create or Update).

I just fixed the example above a little bit to show it.

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to linode_example_fw_block.foo, provider "provider[\"registry.terraform.io/linode/linode\"]" produced an unexpected new value: .example_block_attr: block count changed from 0 to 1.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

config:

terraform {
  required_providers {
    linode = {
      source  = "linode/linode"
    }
  }
}

resource "linode_example_fw_block" "foo" {
}
11

I think this is a similar issue opened open the GitHub Issues page:

12

Hi @zliang-akamai :wave:

I believe you are encountering an analogous issue to that described in Optional Computed Block handling in Plugin Framework.

The schema you have defined is using ListNestedBlock, and the block is effectively Optional. If the configuration is empty, and does not contain a value for example_block_attr, then this will be treated as a zero-length list. Mutating this value during the Create function is not allowed, as this would introduce a discrepancy between the configuration and the planned value, which gives rise to the error that you have encountered:

block count changed from 0 to 1

Effectively, your provider needs a mechanism to store additional values when the example_block_attr length is zero. I believe that the only option you have in this case is to proceed as described in Optional Computed Block handling in Plugin Framework - #4 by bendbennett