How to compare changes for nested block with new framework?

Hello, I have the follow plan and state data in my Update function using the new provider framework…

plan: 
&{Activate:true Comment:"Managed by Terraform" Domain:[{"comment":<null>,"name":"tf-test-pxhjl6z0px-terraform-provider-fastly-framework-2.integralist.co.uk"},{"comment":<null>,"name":"tf-test-pxhjl6z0px-updated-terraform-provider-fastly-framework-1.integralist.co.uk"}] Force:true ID:"123" LastActive:<unknown> Name:"tf-test-pxhjl6z0px-updated" Reuse:<null> Version:<unknown>}

state: 
&{Activate:true Comment:"Managed by Terraform" Domain:[{"comment":<null>,"name":"tf-test-pxhjl6z0px-terraform-provider-fastly-framework-1.integralist.co.uk"},{"comment":<null>,"name":"tf-test-pxhjl6z0px-terraform-provider-fastly-framework-2.integralist.co.uk"}] Force:false ID:"123" LastActive:1 Name:"tf-test-pxhjl6z0px" Reuse:<null> Version:1}

The schema for the domains is:

		Blocks: map[string]schema.Block{
			"domain": schema.SetNestedBlock{
				NestedObject: schema.NestedBlockObject{
					Attributes: map[string]schema.Attribute{
						"comment": schema.StringAttribute{
							MarkdownDescription: "An optional comment about the domain",
							Optional:            true,
						},
						"name": schema.StringAttribute{
							MarkdownDescription: "The domain that this service will respond to. It is important to note that changing this attribute will delete and recreate the resource",
							Required:            true,
						},
					},
				},
			},
		},

In my provider tests I create two domains and then update one of them (I change tf-test-pxhjl6z0px-terraform-provider-fastly-framework-1.integralist.co.uk to include -updated like so tf-test-pxhjl6z0px-updated-terraform-provider-fastly-framework-1.integralist.co.uk.

My API needs to know which specific domain needs updating, so how can I check which domain has been updated?

If the order of the domains were consistent then I could loop over the plan ‘domains’ and for each plan domain I could use the same index to find the domain in the state, but from what I’m seeing the order is not consistent.

My ‘Update’ code is below:

func (r *ServiceVCLResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
	var plan *ServiceVCLResourceModel
	var state *ServiceVCLResourceModel

	// Read Terraform plan data into the model
	resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)

	if resp.Diagnostics.HasError() {
		return
	}

	// Read Terraform state data into the model so it can be compared against plan
	resp.Diagnostics.Append(req.State.Get(ctx, &state)...)

	if resp.Diagnostics.HasError() {
		return
	}

	// NOTE: The plan data doesn't contain computed attributes.
	// So we need to read it from the current state.
	plan.Version = state.Version
	plan.LastActive = state.LastActive

	if !plan.Domain.Equal(state.Domain) {
		for _, domain := range plan.Domain.Elements() {
			v, err := domain.ToTerraformValue(ctx)
			if err != nil {
				tflog.Trace(ctx, "ToTerraformValue error", map[string]any{"err": err})
				resp.Diagnostics.AddError("ToTerraformValue error", fmt.Sprintf("Unable to convert type to Terraform value: %s", err))
				return
			}

			var planDomains map[string]tftypes.Value

			err = v.As(&planDomains)
			if err != nil {
				tflog.Trace(ctx, "As error", map[string]any{"err": err})
				resp.Diagnostics.AddError("As error", fmt.Sprintf("Unable to convert type to Go value: %s", err))
				return
			}

			for _, domain := range state.Domain.Elements() {
				vState, err := domain.ToTerraformValue(ctx)
				if err != nil {
					tflog.Trace(ctx, "ToTerraformValue error", map[string]any{"err": err})
					resp.Diagnostics.AddError("ToTerraformValue error", fmt.Sprintf("Unable to convert type to Terraform value: %s", err))
					return
				}

				var stateDomains map[string]tftypes.Value

				err = vState.As(&stateDomains)
				if err != nil {
					tflog.Trace(ctx, "As error", map[string]any{"err": err})
					resp.Diagnostics.AddError("As error", fmt.Sprintf("Unable to convert type to Go value: %s", err))
					return
				}
			}

			// FIXME: What domain to update? How can I tell which domain has changed?
			var domainToUpdate string
			planDomains["name"].As(&domainToUpdate)

			clientReq := r.client.DomainAPI.UpdateDomain(r.clientCtx, plan.ID.ValueString(), int32(plan.Version.ValueInt64()), domainToUpdate)

			// Update specific aspects of the domain like a 'comment' or 'name'...

			if v, ok := planDomains["comment"]; ok && !v.IsNull() {
				var dst string
				err := v.As(&dst)
				if err != nil {
					tflog.Trace(ctx, "As error", map[string]any{"err": err})
					resp.Diagnostics.AddError("As error", fmt.Sprintf("Unable to convert type to Go value: %s", err))
					return
				}
				clientReq.Comment(dst)
			}

			if v, ok := planDomains["name"]; ok && !v.IsNull() {
				var dst string
				err := v.As(&dst)
				if err != nil {
					tflog.Trace(ctx, "As error", map[string]any{"err": err})
					resp.Diagnostics.AddError("As error", fmt.Sprintf("Unable to convert type to Go value: %s", err))
					return
				}
				clientReq.Name(dst)
			}

			_, httpResp, err := clientReq.Execute()
			if err != nil {
				tflog.Trace(ctx, "Fastly DomainAPI.UpdateDomain error", map[string]any{"http_resp": httpResp})
				resp.Diagnostics.AddError("Client Error", fmt.Sprintf("Unable to update domain, got error: %s", err))
				return
			}
			if httpResp.StatusCode != http.StatusOK {
				tflog.Trace(ctx, "Fastly API error", map[string]any{"http_resp": httpResp})
				resp.Diagnostics.AddError("API Error", fmt.Sprintf("Unsuccessful status code: %s", httpResp.Status))
				return
			}
		}
	}

	clientReq := r.client.ServiceAPI.UpdateService(r.clientCtx, plan.ID.ValueString())
	if !plan.Comment.Equal(state.Comment) {
		clientReq.Comment(plan.Comment.ValueString())
	}
	if !plan.Name.Equal(state.Name) {
		clientReq.Name(plan.Name.ValueString())
	}

	_, httpResp, err := clientReq.Execute()
	if err != nil {
		tflog.Trace(ctx, "Fastly ServiceAPI.UpdateService error", map[string]any{"http_resp": httpResp})
		resp.Diagnostics.AddError("Client Error", fmt.Sprintf("Unable to update service, got error: %s", err))
		return
	}

	// Save updated data into Terraform state
	resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
}

I went digging out of curiosity, and discovered this:

I think what this is saying, is that terraform-plugin-framework does NOT support working with nested blocks in the way you seem to want.

List mode is unsuitable for you because it means blocks are identified by their sequential position in the configuration, meaning adding/deleting one that is not the last, causes all the others to shift, and spurious modifications to be planned.

Set mode is unsuitable for you because all modifications of any kind plan as delete/recreate of the nested blocks.

Map sounds close to what you need. I’m not sure what group mode is.

Ah, I found the descriptions of the other modes:

Thanks @maxb I really appreciate your insights and feedback on the various questions I’ve asked the past week (I see you’ve replied to my other questions and will review in more detail tonight).

Hi @maxb

So one issue I’m having is that the new framework doesn’t provide a HasChanges feature like the v2 SDK.

My API doesn’t return a unique ID for each ‘domain’. The domain is just a ‘name’ (e.g. example.com) and a ‘comment’ (e.g. “this is my production domain”).

Now when it comes to the provider’s Update function I can’t tell which domain has been updated because if I have a test that creates two domains…

  1. example-a.com
  2. example-b.com

…and then in my ‘update’ test I change example-b.com to example-c.com, then when the provider’s Update function is run and I’m looping over the plan data, then I’ll iterate over two available domains and the provider doesn’t know that the example-b.com was renamed to example-c.com as it has no way to know the value had changed unless I compare it to the prior state data.

But even then, I can’t tell from the prior state data whether the example-b.com was deleted and a new example-c.com was created or if it was actually renamed.

So to work around that I’ve added a computed attribute ID to the domain schema and when creating a domain I generate a uuid for the domain, and then in the Update function I’m now able to check if the uuid in of the domain in the plan matches the uuid of the domain in the prior state, and from there I can trigger the relevant API call to update that specific domain.

Now this isn’t ideal as when the provider tests run terraform refresh and the Read function is called my Read function also has to create a new uuid for the domain (as it’s calling the API to get all domains and then just updates the Domain attribute with the returned data), and this causes me to have to use ExpectNonEmptyPlan in the tests otherwise the tests will fail because of the fact that the ID attribute is computed and so this causes the set data type to think the whole thing has changed.

I guess I’d need to change from a nested block set to a MapNestedAttribute but that breaks the interface from prior releases of my provider.

How does your existing provider handle this scenario? :

  1. User performs initial apply with configuration
resource "something_something" "something" {
  domain {
    name = "example-a.com"
  }
  domain {
    name = "example-b.com"
  }
  domain {
    name = "example-c.com"
  }
}
  1. User changes config and performs second apply with configuration
resource "something_something" "something" {
  domain {
    name = "example-b.com"
  }
  domain {
    name = "example-c.com"
  }
}

The only diffs that I’ve been able to get Terraform to display are:

  • Delete all three existing domains and recreate the two in the second config (with SetNestedBlock)
  • Modify the existing example-a.com in place to now be example-b.com; modify the existing example-b.com in place to now be example-c.com; delete the old example-c.com (with ListNestedBlock)

I can’t see any way using nested blocks, that would result in Terraform generating the desired diff of deleting example-a.com and leaving the other two intact.

I do not see how this workaround can actually work. Your generated UUIDs are saved in the state, yes, but they are not in the user’s .tf files, so you still can’t tell the difference between a delete and add, or a rename - because either scenario results in the user re-executing Terraform with the same input.

Regarding HasChange, Framework's alternative to HasChange() - #2 by bflad

Correct my implementation doesn’t work.

I’ve reviewed the original v2 SDK provider and it seems we added custom logic for determining whether a resource was modified/added/deleted…

So I’m going to have to implement something similar it seems.

But to be clear the diffs for sets were always confusing to users because of the entire set being shown as deleted/recreated.

Yeah this is the biggest issue our original provider had :frowning:

Wondering if there is a better type to use to avoid this issue?

A set is out of the question (and so a block is too as it only supports list/set types) but maybe a nested attribute instead of a block and then a map type could be used.

The problem with a map is that visually (from a user experience perspective) is that a map attribute looks to be quite verbose.

A block set is quite clean, which is probably why the original provider was built using them.

An interesting option could be to use a block map - which would let you write things like:

resource "something_something" "something" {
  domain "example-a.com" {}
  domain "example-b.com" {
    description = "This one is the second one"
  }
  domain "example-c.com" {
    special = true
  }
}

and then Terraform CLI would correctly identify and display diffs for particular domains being added, removed, etc.

This is the feature that is not implemented in terraform-plugin-framework … but I managed to get it working well enough to test, by setting up my test provider to use a forked version of the framework, and about 15 minutes of frantic copy/pasting from the equivalent implementation for list nested blocks.

This provides an elegant syntax for users… unless they decide they want to calculate the domains dynamically, in which case it turns into this mess:

resource "something_something" "something" {
  dynamic "domain" {
    for_each = toset(["example-a.com", "example-b.com", "example-c.com"])
    labels   = [domain.key]
    content {
      description = "Dynamic from ${domain.value}"
    }
  }
}

Considering Terraform core already has support for these block maps, maybe it’s worth trying to convince the terraform-plugin-framework maintainers to support the feature too?