Terraform Plugin Framework - Dynamic Resource Creation

Terraform Providers Plugin Development
1

Hi Everyone,

I am looking into possible solutions of dynamically generating resources and data-sources based on meta information provided by a product. A template based approach could be a solution to generate the provider, but this would mean for every version of the product the provider needs to be regenerated from templates. Since meta information (json format) is available via the API of the product I am considering generating structs and schemas used for resources and data-sources based on this meta information. There are however a few things blocking me that I would like to discuss and or get your opinion on:

  1. Reading plan data into the model, or getting and saving data into Terraform state requires a struct to be passed to the corresponding function. When this struct is generated with the function reflect.StructOf(), values are not mapped correctly and errors are triggered. Executing a type assertion of reflect.Value in the Into() function in internal/reflect/into.go and the FromValue() function in internal/reflect/outof.go allows for the values to be mapped correctly. Is it a design choice to not allow structs that are created reflect.StructOf()? Would it be a possibility to add type assertion into these functions?

  2. During my investigation I have temporarily embedded the meta files in the provider, which allowed me to try dynamic struct creation with the reflect.StructOf() function. From a proof of concept point of view this seems to work. Ideally I would now like to request the meta files from the api of the product, thus removing the need of embedding of meta files. In this case I would need the provider configuration details to connect to the api for meta data retrieval. During the creation of the resource functions ( same applies for data sources ) the provider configuration is not available and thus I am unable to retrieve the meta information. Is there a way to access provider configuration during this phase?

Kind regards,

2

I think other providers who construct the resources & data sources dynamically are doing so at build time rather than at run time. The advantage of that is that the capabilities are static for a particular provider version, meaning the user has more control - previously working code shouldn’t just break due to the provider now expecting a slightly different set of parameters.

3

Hi Stuart,

Thank you for your reply, I have indeed seen that other providers are doing so at build time. I understand the advantage that you are highlighting. Would this undesired behaviour not also apply to any “breaking” change in behaviour between provider versions?

In the case where a product version is mapped to a specific provider version, if a product upgrade is executed the provider version would need to change accordingly to match all the functionality. This could also introduce breaking changes to previously working code of the user. Which could mean that an “older” version of the provider is not providing the desired functionality anymore or that the product has changed behaviour for existing functionality. What is your view on these type of situations, which should be avoided but are not always in my scope of control.