Delete element from list during update

Terraform Providers Plugin Development
1

Hello,
I have this imported

resource "my-resource" "test" {
  id                  = 1234
  name                = "test"
  my_list = [
    {
      id            = 5678
      sub_obj      = {
        id          = "this-is-id"
      }
    },
    {
      id            = 9123
      sub_obj      = {
        id          = "this-is-id2"
      }
    },
  ]
}

I want to be able to remove elements from my_list by writing for example.

resource "my-resource" "test" {
  id                  = 1234
  name                = "test"
  my_list = [
    {
      id            = 5678
      sub_obj      = {
        id          = "this-is-id"
      }
    },
  ]
}

My schema is

resp.Schema = schema.Schema{
		Attributes: map[string]schema.Attribute{
		...
		"my_list": schema.ListNestedAttribute{
			Computed: true,
			Optional: true,
			NestedObject: schema.NestedAttributeObject{
				Attributes: map[string]schema.Attribute{
					"id": schema.Int64Attribute{
						Computed: true,
					},
					"sub_obj": schema.ObjectAttribute{
						Computed: true,
						AttributeTypes: map[string]attr.Type{
							"id":          types.StringType,
						},
					},
				},
			},
		},
	},
	}

On Update() I started writing this but I’m a bit lost, see comments

func (r *myResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
// Here I have multiple different update methods depending on what I'm updating. 
// So do I have a better choice than getting plan & state and comparing ? 
// I saw in tutorial only plan use, but then we provide all the plan values, 
// may they be updated or not
	var plan, state myResourceModel
	resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
	resp.Diagnostics.Append(req.State.Get(ctx, &state)...)

// Do I have a better way than comparing the elements like this?
	if !plan.MyList.Equal(state.MyList) {
		for _, planEl := range plan.MyList.Elements() {
			found := false
			for _, stateEl := range state.MyList.Elements() {
				if planEl.Equal(stateEl) {
					found = true
				}
			}
			if !found {
				_, errDiag := types.ObjectValueFrom(ctx, MyType, planEl)
				resp.Diagnostics.Append(errDiag...)
// How to marshal/map correctly to access 
// MyList.ID / MyList.SubObj.ID values here?

			}
		}
	}
	resp.Diagnostics.Append(resp.State.Set(ctx, state)...)
}

Always a bit lost with the lists when writing the provider… took me a good while to be able to have this complex list read correctly in resources, and it was with the help of browsing existing complex provider repos on github. I feel the tutorial is a bit light on the complex parts of things

Thank you.

2

Hi @Khyme :wave:

The first thing I notice, is that both the my_list.id and my_list.sub_obj attributes are marked as Computed. With this schema you will be unable to supply values in the way in which you have illustrated in the Terraform configuration for these attributes as they are considered “read only”, and the value assigned must be managed by the provider itself. If you try to set values for these attributes in the configuration you will see an error along the following lines:

│ Error: Invalid Configuration for Read-Only Attribute
│ 
│   with example_resource.example,
│   on resource.tf line 9, in resource "example_resource" "example":
│    9: resource "example_resource" "example" {
│ 
│ Cannot set value for this attribute as the provider has marked it as read-only. Remove the configuration line setting the value.
│ 
│ Refer to the provider documentation or contact the provider developers for additional information about configurable and read-only attributes that are supported.

I’ve modified the schema to add Optional: true to both my_list.id and my_list.sub_obj, and set-up the CRUD functions as follows:

type exampleResource struct {
}

func NewResource() resource.Resource {
	return &exampleResource{}
}

func (e *exampleResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
	resp.TypeName = req.ProviderTypeName + "_resource"
}

func (e *exampleResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
	resp.Schema = schema.Schema{
		Attributes: map[string]schema.Attribute{
			"my_list": schema.ListNestedAttribute{
				Computed: true,
				Optional: true,
				NestedObject: schema.NestedAttributeObject{
					Attributes: map[string]schema.Attribute{
						"id": schema.Int64Attribute{
							Computed: true,
							Optional: true,
						},
						"sub_obj": schema.ObjectAttribute{
							Computed: true,
							Optional: true,
							AttributeTypes: map[string]attr.Type{
								"id": types.StringType,
							},
						},
					},
				},
			},
		},
	}
}

type exampleResourceData struct {
	MyList types.List `tfsdk:"my_list"`
}

func (e *exampleResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
	var data exampleResourceData

	diags := req.Plan.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)
}

func (e *exampleResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
	var data exampleResourceData

	diags := req.State.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)
}

func (e *exampleResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
	var data exampleResourceData

	diags := req.Plan.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)
}

func (e *exampleResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
	var data exampleResourceData

	diags := req.State.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}
}

Using the following Terraform config, and executing terraform apply, I see the following:

resource "example_resource" "example" {
  my_list = [
    {
      id      = 5678
      sub_obj = {
        id = "this-is-id"
      }
    },
    {
      id      = 9123
      sub_obj = {
        id = "this-is-id2"
      }
    },
  ]
}

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # example_resource.example will be created
  + resource "example_resource" "example" {
      + my_list = [
          + {
              + id      = 5678
              + sub_obj = {
                  + id = "this-is-id"
                }
            },
          + {
              + id      = 9123
              + sub_obj = {
                  + id = "this-is-id2"
                }
            },
        ]
    }

Plan: 1 to add, 0 to change, 0 to destroy.
example_resource.example: Creating...
example_resource.example: Creation complete after 0s

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

If I then modify the configuration and run terraform apply again I see the following:

resource "example_resource" "example" {
  my_list = [
    {
      id      = 5678
      sub_obj = {
        id = "this-is-id"
      }
    }
  ]
}

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # example_resource.example will be updated in-place
  ~ resource "example_resource" "example" {
      ~ my_list = [
          - {
              - id      = 9123 -> null
              - sub_obj = {
                  - id = "this-is-id2"
                } -> null
            },
            # (1 unchanged element hidden)
        ]
    }

Plan: 0 to add, 1 to change, 0 to destroy.
example_resource.example: Modifying...
example_resource.example: Modifications complete after 0s

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Can you confirm if both my_list.id and my_list.sub_obj attributes must be “computed-only”? If so, perhaps you can supply the details of the CRUD functions and the import function that you have in place currently and we can go from there.

3

Thank you for your response, I thought only the parent needed to be optional.

Now I still have an issue as to how to access the different properties of my object in order to provide them to my API, see my comments in code

func (r *myResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
// Here I have multiple different update methods depending 
// on what I'm updating. 
// So do I have a better choice than getting plan & 
// state and comparing ? 
// I saw in tutorial only plan use, but then 
// we provide all the plan values to the Update client method, 
// may they be updated or not. I think in my case where I call different 
// client methods depending on what I update from my object
// I must unmarshal plan & state
	var plan, state myResourceModel
	resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
	resp.Diagnostics.Append(req.State.Get(ctx, &state)...)

// Do I have a better way than comparing the elements like this?
	if !plan.MyList.Equal(state.MyList) {
		for _, planEl := range plan.MyList.Elements() {
			found := false
			for _, stateEl := range state.MyList.Elements() {
				if planEl.Equal(stateEl) {
					found = true
				}
			}
			if !found {
				_, errDiag := types.ObjectValueFrom(ctx, MyType, planEl)
				resp.Diagnostics.Append(errDiag...)
      // How to marshal/map correctly to access 
      // MyList.ID / MyList.SubObj.ID values here?
      r.client.DeleteByID(ctx, obj.Attributes()["id"]) // API is something like this
			}
		}
	}
	resp.Diagnostics.Append(resp.State.Set(ctx, state)...)
}
4

Okay I managed to make it work using something like this

	for _, planEl := range plan.MyList.Elements() {
					if planEl.Equal(stateEl) {
						found = true
					}
				}
				obj, errDiag := types.ObjectValueFrom(ctx, MyType, stateEl)
				id := obj.Attributes()["id"].(types.Int64).ValueInt64()

So I just wonder about structure / code best practices now

5

Hi @Khyme,

You can also unmarshal the object elements within MyList using something like the following:

type exampleResourceData struct {
	MyList types.List `tfsdk:"my_list"`
}

type myListElementstruct struct {
	ID     types.Int64  `tfsdk:"id"`
	SubObj types.Object `tfsdk:"sub_obj"`
}

func (e *exampleResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
	var data exampleResourceData

	diags := req.Plan.Get(ctx, &data)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	var elem []myListElementstruct

	diags = data.MyList.ElementsAs(ctx, &elem, false)
	resp.Diagnostics.Append(diags...)

	if resp.Diagnostics.HasError() {
		return
	}

	for _, v := range elem {
		attrs := v.SubObj.Attributes()
		// operate on attrs (e.g., 
	}

	diags = resp.State.Set(ctx, &data)
	resp.Diagnostics.Append(diags...)
}

You might also want to consider using a single nested attribute instead of object attribute - https://developer.hashicorp.com/terraform/plugin/framework/handling-data/attributes/object

In terms of the “structure / code best practices”. Can you expand on why you “have different update methods depending on what I’m updating”. Can you also describe why you are comparing state and plan values in the update method?

6

Can you expand on why you “have different update methods depending on what I’m updating”.

If I update the name of my object, I’m calling a Rename() client function. If I delete an element of my list, I’m calling a DeleteElement() client function.
I mean that I cannot simply forward the new plan model to an Update() API that would appropriately update all fields, it’s made of multiple independent API calls for the different bits.

Can you also describe why you are comparing state and plan values in the update method?

How else can I know which elements were deleted and which were added at planning?

7

Hi Khyme,

Thank you for the clarification. Given your use case then you could compare plan and state values as you have described (here is another example - Framework's alternative to HasChange() - #2 by bflad).

Another consideration in your provider design is whether it is possible to separate out resource object management to reduce the number of potential API calls required during an Update function call and provide a logical separation between resources. For instance, are the sub_obj that you have in your configuration separate resources? If so, then another potential design might be to manage sub_obj separately, for example:

resource "subobj_resource" "one" {
  id = "this-is-id"
}

resource "subobj_resource" "two" {
  id = "this-is-id2"
}

 resource "example_resource" "example" {
  my_list = [
    {
      id      = 5678
      sub_obj = {
        id = subobj_resource.one.id
      }
    },
    {
      id      = 9123
      sub_obj = {
        id = subobj_resource.two.id
      }
    },
  ]
}

Under these circumstances, the deletion of the second subobj, in the configuration example you provided, would be handled by removing resource "subobj_resource" "two" from configuration, and the Update function within resource "example_resource" "example" would only need to account for updating the plan to reflect this removal (using the state and plan comparison you described), rather than also needing to call DeleteElement().

Modelling how a Terraform provider should map onto API endpoints is nuanced and varies cases-by-case, but generally speaking, if entities/objects can be managed through individual resources and then referred to by other resources then this is a pattern that is often applied to simplify the logic. An example of this kind of design can be seen within the AWS provider, such as the usage of security groups and security group rules. Given this, perhaps an additional consideration should be whether to invert the relationship between sub_obj_resource and example_resource along the same lines as AWS security gorups. For instance, if the relationship between example_resource and sub_obj_resource is a “has-a” relationship, then perhaps this could be represented by:

resource "example_resource" "example" {
  /* ... */
}

resource "subobj_resource" "one" {
  id = "this-is-id"
  example_resource_id = example_resource.one.id
}

resource "subobj_resource" "two" {
  id = "this-is-id2"
  example_resource_id = example_resource.one.id
}
8

Thank you.
I guess I should go with the first approach but first I want this to work :slight_smile:

So with your help, I thought I had it but after all not.

Here is my PR as it might be easier Kim/vpc by Khyme · Pull Request #159 · timescale/terraform-provider-timescale · GitHub
My adding / removing of items works right, on my API end its all right.

But, I have a readonly field, peer_cidr.

"peer_cidr": schema.StringAttribute{
	Computed: true,
	PlanModifiers: []planmodifier.String{
		stringplanmodifier.UseStateForUnknown(),
	},
},

It’s not set when I add an element and its actually always empty for now. But it’s always marked known after apply, so every time I do terraform apply, the one in my tf file is not considered equal to the one in the state, so it creates a new one.
I can only set a string default, default can’t be null.
Do I have no choice but set it to empty string instead of null ?