Problems using CustomizeDiff with calculated fields

Hi,

I hope someone can help me. I’m having some problems using CustomizeDiff with calculated fields in that it panics due to a nil pointer.

My CustomizeDiff function looks like this:

		CustomizeDiff: func(ctx context.Context, d *schema.ResourceDiff, meta interface{}) error {
			old, new := d.GetChange("network_interface")

			oldList := old.([]interface{})
			newList := new.([]interface{})

			// If the new list is shorter than the old list we
			// want to force new.
			if len(newList) < len(oldList) {
				d.ForceNew("network_interface")
				return nil
			}

			for i, oi := range oldList {
				oiB := oi.(map[string]interface{})
				niB := newList[i].(map[string]interface{})

				if oiB["type"] != niB["type"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.type", i))
				}
				if oiB["ip_address_family"] != niB["ip_address_family"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.ip_address_family", i))
				}
				if oiB["network"] != niB["network"] {
					d.ForceNew(fmt.Sprintf("network_interface.%d.network", i))
				}
			}

			return nil
		},

In essence, it is attempting to detect a few scenarios where we know we need to re-create the resource: a) when we’ve removed an item from the list b) any of the items in the list have certain fields changed c) the order of the list has changed.

This code mostly works in manual and automated tests. The network field is the ID of another resource. If the creation of that resource happens during the same plan/apply as changing the value of network then the provider crashes:

2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: panic: runtime error: invalid memory address or nil pointer dereference
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xabbf50]
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: 
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: goroutine 54 [running]:
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: github.com/hashicorp/terraform-plugin-sdk/v2/internal/helper/plugin.(*GRPCProviderServer).PlanResourceChange(0xc00000e280, 0xd7ae20, 0xc0000813c0, 0xc00021c000, 0xc00000e280, 0xc00000e290, 0xc90498)
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/internal/helper/plugin/grpc_provider.go:797 +0x1110
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5._Provider_PlanResourceChange_Handler.func1(0xd7ae20, 0xc0000813c0, 0xc22080, 0xc00021c000, 0xc0000813c0, 0xba1e00, 0xc00009cd01, 0xc0003e4d80)
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5/tfplugin5.pb.go:3294 +0x86
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: github.com/hashicorp/terraform-plugin-sdk/v2/plugin.Serve.func3.1(0xd7aee0, 0xc00057e390, 0xc22080, 0xc00021c000, 0xc0003e4d60, 0xc0003e4d80, 0xc000521ba0, 0x526718, 0xbf8f40, 0xc00057e390)
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/plugin/serve.go:76 +0x87
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5._Provider_PlanResourceChange_Handler(0xc30860, 0xc00000e280, 0xd7aee0, 0xc00057e390, 0xc00009cde0, 0xc000346d40, 0xd7aee0, 0xc00057e390, 0xc000447000, 0x7d5)
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/internal/tfplugin5/tfplugin5.pb.go:3296 +0x14b
2020-08-23T21:17:56.335+0100 [DEBUG] plugin.terraform-provider-x: google.golang.org/grpc.(*Server).processUnaryRPC(0xc000268a80, 0xd836a0, 0xc000107b00, 0xc000188600, 0xc000192c30, 0x129d168, 0x0, 0x0, 0x0)
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/google.golang.org/grpc/server.go:1171 +0x50a
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: google.golang.org/grpc.(*Server).handleStream(0xc000268a80, 0xd836a0, 0xc000107b00, 0xc000188600, 0x0)
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/google.golang.org/grpc/server.go:1494 +0xccd
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc00012a360, 0xc000268a80, 0xd836a0, 0xc000107b00, 0xc000188600)
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/google.golang.org/grpc/server.go:834 +0xa1
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: created by google.golang.org/grpc.(*Server).serveStreams.func1
2020-08-23T21:17:56.336+0100 [DEBUG] plugin.terraform-provider-x: 	/home/stuart/Projects/terraform-provider-x/vendor/google.golang.org/grpc/server.go:832 +0x204

The crash appears to be happening in line 796 of grpc_provider.go here (at if v.NewExtra != nil {):

	newExtra := map[string]interface{}{}

	for k, v := range diff.Attributes {
		if v.NewExtra != nil {
			newExtra[k] = v.NewExtra
		}
	}
	privateMap[newExtraKey] = newExtra

Sticking in a debug Printf I can see that the value of network is nil:

2020/08/23 21:17:56 [DEBUG]: diff.Attributes: map[string]*terraform.ResourceAttrDiff{"network_interface.1.network":<nil>}

Am I doing something wrong in CustomizeDiff to cause this? Has anyone come across something like this before?

Any help is appreciated!

Hi @yoink00! Sorry that didn’t work as expected.

I think you’ve found a bug in the Terraform SDK here. I’m not sure exactly what’s happening but I wonder if d.ForceNew isn’t correctly handling those indexed paths; I think other uses of CustomizeDiff I’ve seen have tended to use ForceNew mainly with top-level attributes, rather than with attributes in nested schema.Resource.

As a workaround for now I might try just always using the top-level network_interface as the ForceNew, rather than the specific attributes inside. This will make the resulting plan less accurate in the UI, but will hopefully avoid the bug until it can be fixed.

I’d also suggest reporting an issue in the Terraform Plugin SDK repository, so the SDK development team will be aware of it. As well as hopefully eventually fixing it, they might also be able to offer a different workaround due to their greater familiarity with the SDK codebase.

Hi @apparentlymart,

Thank you for your response.

I tried your suggestion but I get a panic at exactly the same point.

I will raise an issue as you suggest.

Again, thank you for your help.