Hi all 
,
The Terraform DevEx team has recently released a tech preview that’s aiming to put some guardrails around the provider code generation problem space. This isn’t intended to be an “all-in-one” code generator, but rather general pieces that can fit together to build a code generation solution that works for specific providers.
Here, the problem is broken into 3 pieces, with an implementation to showcase potential solutions for each. Developers may be able to get value out of all the solutions, or maybe just a few of them, but hopefully it will reduce the amount of custom code generation a maintainer needs to write.
The blog post and documentation link above contain a lot of this info, but I’ll summarize the problem and solutions here as well. Here is a high-level summary of the design:
1. Representing a Terraform provider
In order to break this problem into pieces, we first need an intermediate representation of a Terraform provider to separate some concerns. We created the provider code specification to serve this purpose, which is just a JSON schema that defines how a Terraform provider is modeled.
If you’re already familiar with provider development you’ll notice it’s mostly schema information at the moment, with some custom Go bits like imports.
The full JSON schema definition is here.
2. Translating from source(s) to a Terraform provider
A “source” most commonly refers to an IDL like an OpenAPI specification or Protobuf, but a source doesn’t necessarily need to be an IDL. There also may be more than one source, could be multiple OpenAPI specs, a protobuf file and an API SDK, etc.
A provider spec generator takes in one or more sources and generates a provider code specification. This process needs to translate how a source, like an OpenAPI specification, maps to Terraform’s concepts such as data handling. This problem ranges in complexity depending on what source(s) your provider’s API uses.
For a source like Cloud Formation Resource Schema, the semantics of resources map pretty well to Terraform’s data handling concepts, making the translation process easier. The awscc provider is generated based on this source today.
For a more generic source like OpenAPI, there is more complexity/customization in determining the proper mapping to a Terraform provider. We built the OpenAPI provider spec generator that showcases one potential solution for translating this kind of source. This solution relies on a configuration file to drive that customization.
3. Generating Terraform provider stuff
Once you have a provider code specification, you can start generating the Terraform provider. This most likely will be generating actual Terraform provider Go code, but isn’t necessarily limited to just this. You could also use a provider code specification to generate tests, documentation, etc.
We built the Plugin Framework code generator that generates the Terraform SDK Go code for managed resources, data source, or a provider. As the name suggests, it generates Go code that uses the Plugin Framework SDK and currently it focuses on generating schema code.
Putting the pieces together
We wrote a workflow example that shows a sample workflow for how these tools could be used together. Currently it’s CLI focused, but there are no technical restrictions that prevent more CI friendly approaches.