Terraform-plugin-framework codebase has extreme levels of code duplication - can it be maintained long term?

maxb · January 7, 2023, 10:45pm

I have been learning about terraform-plugin-framework, and trying to add a missing feature.

I was surprised to find, that despite being a newly written library, it seems to have extreme levels of copy/paste, with no obvious sign of code generation in use to maintain control of the repetitive code.

Example:

I took as a sample, a particular line of documentation comment text, In Terraform 1.2.7 and later, this warning diagnostic is displayed, and searched the codebase - this comment is duplicated 48 times throughout the project.

Imagine the pain of making and reviewing a PR, if a piece of documentation like this needed to be updated in any way!

DRY (Don’t Repeat Yourself) is a fairly well-known programming principle, which terraform-plugin-framework appears to be vigourously rejecting - it makes me worry about whether it can be a maintainable project in the long term.

Is anyone able to share some insight into why it is the way it is?

bflad · January 9, 2023, 5:36pm

Hi @maxb Thank you for raising this topic. As the author of a lot of that code duplication, I can hopefully provide you the insight you are looking for here.

The Go documentation comment you mention here is present in the schema handling and its important from the maintainer’s perspective that the Go comments are fully realized for every type and field, so it works well with the Go package documentation and editor/LSP support for developers to easily access the information.

The history here is that originally in the framework’s design, there was a single tfsdk.Attribute type which was an overloaded abstraction for representing an attribute across all the provider schema concepts (provider attribute, provider meta attribute, resource attribute, and data source attributes), very similar to terraform-plugin-sdk. This has always been relatively problematic for provider developers since not all behaviors and capabilities should be code-able for each concept. To address this as part of Consider Splitting tfsdk.Attribute into datasource.Attribute, provider.Attribute, and resource.Attribute · Issue #132 · hashicorp/terraform-plugin-framework · GitHub, the schema handling abstraction was split into individual Go packages to align only the available capabilities for each concept. Due to pre-1.0 feedback, the attributes were further split to be typed implementations of an interface (e.g. StringAttribute implementing Attribute interface), which caused a larger repeatable surface area. To provide some additional hidden context, this effort was being completed quickly before a self-imposed 1.0 release timeline in December, since getting the breaking changes in was most important at the time.

So with all that said, this particularly duplicated Go code for schemas is probably less than ideal from a framework code update standpoint at the moment, but its logic is quite simple in each case meaning that it probably is okay for Go code generation or potentially generics. I’ve developed similar code generation within terraform-provider-aws when I was maintaining that project, so its not for lack of knowledge of how to do it, just more relating to the development time aspect. The main “challenge” for this area of code is that we do want to ensure that every schema concept (provider versus resource, etc.) is able to customize its Go comments since Terraform may apply different behaviors in each concept. Not difficult, just something to consider.

I’ve created a followup issue here: Consider Go Code Generation for Schema Types · Issue #612 · hashicorp/terraform-plugin-framework · GitHub

maxb · January 10, 2023, 8:05am

Using code generation for the schema types would be a great start!

For context, I would like to implement the two block nesting modes which are missing from the framework, map and group:

github.com

hashicorp/terraform-plugin-framework/blob/6139030b2d0ed6df585faf8682310746b147bbfa/internal/fwschema/block_nested_mode.go#L6-L10


      
          // While the protocol and theoretically Terraform itself support map and group
          // nesting modes, this framework intentionally only supports list, set, and
          // single blocks as those other modes were not typically implemented or
          // tested with Terraform since the older Terraform Plugin SDK did not support
          // them.

(It says intentionally unimplemented… I hope I can convince you that it’s useful functionality already there in Terraform core, that requires minimal work to expose.)

I have already got it working, but the amount of copy/paste I needed to do, to do so, got me questioning things.

The schema packages were indeed a substantial part of it, but there is a lot of repetitive code here too:

github.com

hashicorp/terraform-plugin-framework/blob/main/internal/fwserver/block_plan_modification.go

package fwserver

import (
	"context"
	"fmt"

	"github.com/hashicorp/terraform-plugin-framework/internal/fwschema"
	"github.com/hashicorp/terraform-plugin-framework/internal/fwschema/fwxschema"
	"github.com/hashicorp/terraform-plugin-framework/internal/fwschemadata"
	"github.com/hashicorp/terraform-plugin-framework/internal/logging"
	"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
	"github.com/hashicorp/terraform-plugin-framework/types"
	"github.com/hashicorp/terraform-plugin-framework/types/basetypes"
)

// BlockModifyPlan performs all Block plan modification.
//
// TODO: Clean up this abstraction back into an internal Block type method.
// The extra Block parameter is a carry-over of creating the proto6server
// package from the tfsdk package and not wanting to export the method.

This file has been truncated. show original

github.com

hashicorp/terraform-plugin-framework/blob/main/internal/fwserver/block_validation.go

package fwserver

import (
	"context"
	"fmt"

	"github.com/hashicorp/terraform-plugin-framework/internal/fwschema"
	"github.com/hashicorp/terraform-plugin-framework/internal/fwschema/fwxschema"
	"github.com/hashicorp/terraform-plugin-framework/internal/fwschemadata"
	"github.com/hashicorp/terraform-plugin-framework/internal/logging"
	"github.com/hashicorp/terraform-plugin-framework/schema/validator"
	"github.com/hashicorp/terraform-plugin-framework/types/basetypes"
)

// BlockValidate performs all Block validation.
//
// TODO: Clean up this abstraction back into an internal Block type method.
// The extra Block parameter is a carry-over of creating the proto6server
// package from the tfsdk package and not wanting to export the method.
// Reference: https://github.com/hashicorp/terraform-plugin-framework/issues/365

This file has been truncated. show original

bflad · January 10, 2023, 12:20pm

You are welcome to raise a feature request issue for further discussion on the codebase (Issues · hashicorp/terraform-plugin-framework · GitHub), however the reality behind why they are not implemented is for multiple reasons:

Providers should prefer nested attributes over blocks going forward. Nested attributes are more practitioner-friendly as they allow configuration via normal expressions, such as for statements, instead of the less friendly Dynamic Blocks - Configuration Language | Terraform | HashiCorp Developer.
The map and group block nesting modes are likely not implemented wholly, or correctly, within Terraform core since no supported higher level SDK enabled providers to include them so there is limited testing of configuration, plan, and state handling for them. These issues could be practitioner-facing panics. Given that these fixes would require practitioners to update to latest Terraform version releases, this limits the utility of enabling the functionality when equivalent and preferable functionality exists. If I’m remembering correctly, map nested attributes have not had major issues in Terraform version 1.0.3 onwards.
This decision was in coordination with the Terraform maintainers to not enable them for the reasons above, even though as you found out it would be relatively straightforward in the framework code to enable them.

Is there functionality you are looking for that is not available with map nested attributes?

Topic		Replies	Views
Terraform-plugin-codegen-framework best practices Plugin Development	1	247	December 11, 2023
TF custom provider documentation is a challenge Plugin Development	3	385	October 4, 2021
Announcing the Terraform Plugin Framework 1.0 Plugin Development	0	342	December 14, 2022
Creating schema for data source using terraform plugin framework Plugin Development	0	255	July 11, 2023
Is the code generation framework dead? Plugin Development	1	35	February 21, 2025

Terraform-plugin-framework codebase has extreme levels of code duplication - can it be maintained long term?

Related topics