Invoke Provider's Configure or ValidateConfig method manually

Terraform Providers Plugin Development
1

I wrote my first provider containing provider defined functions, but ran into an issue with the configuration context not always being available.

The function I wrote generate_resource_name depends on the provider configuration.

When invoking the function from Terraform, the configuration context isn’t yet always available as the Configure method is performed later on and in a separate process.

To overcome this issue I’ve moved the Configure logic to the method ValidateConfig. As Terraform invokes (RPC) the provider multiple times actually, the ValidateConfig method for building the provider’s context wasn’t process (or thread safe).
Therefor I implemented a file-caching method for the configuration context to make it process-safe.
The way I was able to at least get the ValidateConfig method to be invoked before invoking functions, was by making sure my users implement a datasource object within their provider.tf file. This datasource triggers the provider’s ValidateConfig.

But I’m not happy with the current implementation as it leaves potential clutter behind, and might result into potential issues I guess when using this provider multiple times across modules and submodules in a single workspace. First question people also often ask… why do I need this datasource, apart from a lot of people forgetting this step (overlooking the “obvious” docs I generated :p)

I would like to know if there would be a better method to handle this, so the configuration can be available to a static function invocation. I never figured out how-to do so, but I guess it should also be possible to invoke the Configure or ValidateConfig method somehow manually when invoking a function?

2

Hey there @ThomasGeens, I have a similar answer over in a closed issue that probably isn’t easy to find, but I’ll link it here since it’s all still relevant: Cannot define provider functions that are dependent on the provider configuration · Issue #1093 · hashicorp/terraform-plugin-framework · GitHub

The TLDR is that it’s by design that provider configuration is not meant to be available for provider functions, so we don’t have a Configure method for them at all.

As far as those methods being invoked, Terraform won’t invoke RPC’s like ValidateProviderConfig ((Provider).ValidateConfig) or ConfigureProvider (Configure) unless it’s actually needed. If you’re running terraform validate, then Terraform will not configure the provider, since that operation is also an “offline operation”.

1 Like
3

That being said, Terraform providers in general are arbitrary code, so we can’t stop the provider from just doing whatever they want in their (Function).Run code, we can only describe/enforce the design expectations of Terraform and the practitioners who are running it.

4

Hi @austin.valle, thanks for your quick response already. The Github issue which you mentioned is what actually forced me to implement the caching method via the ValidateConfig RPC :slight_smile: But… far from ideal the overhead I bloated out.

I follow what you’re saying on the design principles. But still, in my case I’m writing a provider for “online” purposes only, apart from TF’s builtin validation flows etc. And without any requirements for Datasources or Resources, this renders the configuration for a provider useless in my situation then.

As the ValidateConfig method is “allowed” by the current design to be invoked for Dataresources or Resources during “offline” phases, I still don’t truly understand why it shouldn’t be allowed to be invoked upfront for functions as well. It’s the same with the “Configure” method for example, it also gets invoked during previous “offline” iterations when the provider is not even “online” yet, and there is warning of this mentioned in the docs everywhere, same could’ve been done for functions. So based on this I don’t understand why the provider’s behaviour is completely different for functions.
In my situation it could help if the ValidateConfig method was invoked upfront :slight_smile: then I wouldn’t have to cache the configuration context and have it mutex threadsafe stored instead.

5

Had a short discussion with my copilot about it :smiley: Don’t you just love those new reasoning models :heart_eyes:

So based on the discussion below my literal question would be:

  • Can Terraform be modified to perform a ValidateProviderConfig RPC before a CallFunction RPC?
    So basically implement the same validation for functions regarding the invocation of ValidateProviderConfig as with datasources?

Copilot discussion

Would it be possible for me to simply just invoke the ValidateConfig method myself when a CallFunction is being done?

You’re observing a known characteristic of how Terraform interacts with providers, especially concerning provider-defined functions in scenarios like terraform validate where no resources or data sources explicitly invoke the provider’s configuration lifecycle.

While you could technically call the ValidateConfig method from your GenerateResourceNameFunction.Run method (if the function had a reference to the provider instance), it wouldn’t work as intended for the purpose of processing the provider’s HCL configuration.

Here’s why:

  1. Missing Configuration Input: The ValidateConfig method has the signature: func (p *resourcenamingtoolFunctionsProvider) ValidateConfig(ctx context.Context, req provider.ValidateConfigRequest, resp *provider.ValidateConfigResponse) The crucial part here is req.Config. This field in provider.ValidateConfigRequest is how Terraform provides the actual HCL configuration block of your provider to the method.
  2. Function’s Run Method Signature: Your function’s Run method (e.g., in GenerateResourceNameFunction) receives function.RunRequest. This request contains the arguments passed to the function in the Terraform configuration, but it does not contain the provider’s own HCL configuration block.

Therefore, even if you called p.ValidateConfig from your function’s Run method, you would not have the necessary provider.ValidateConfigRequest (specifically, the req.Config part) to pass to it. You cannot reconstruct the provider’s configuration block from within the function’s execution context.]

Your current approach of using ValidateConfig to process and cache the configuration to a file (and then having Configure and the function potentially read from this file) is a workaround for the lifecycle behavior you’ve encountered. The resourcenamingtool_status data source helps ensure that ValidateConfig is triggered.

If the Terraform Plugin Framework guaranteed that Configure (which could get configuration from req.Config was always called before any function invocation, even in “offline” states, your file-based workaround might be less necessary. However, given the current behavior you’re working around, directly calling ValidateConfig from the function’s Run method isn’t a viable alternative due to the missing provider configuration input.

And would it be possible by implementing a custom provider server somehow and changing the behaviour, so that when a function gets called the ValidateConfig req.Config is made available.

You’re delving into the fundamental architecture of Terraform plugins and their communication with Terraform Core.

The short answer is that implementing a custom provider server to directly make the req.Config from ValidateConfig (which is provider.ValidateConfigRequest.Config) available at the moment a function is called (i.e., within the CallFunction RPC handler, using its function.RunRequest) is not feasible in a way that would simplify your current model or bypass the existing lifecycle.

Here’s why:

  1. gRPC Protocol Definition: The communication between Terraform Core and your plugin is defined by a gRPC protocol (e.g., tfplugin6).
  • The ValidateProviderConfig RPC (which maps to your ValidateConfig method) is how Terraform Core sends the provider’s HCL configuration to the plugin. The request for this RPC (ValidateProviderConfigRequest) contains the Config field.
  • The CallFunction RPC is separate. Its request (CallFunctionRequest) contains the function name and its arguments. It does not include the provider’s HCL configuration.
  1. Provider Server Role: The provider server (which terraform-plugin-go helps you run) is an implementation of this gRPC service. It responds to RPCs sent by Terraform Core.
  • When Terraform Core invokes a function, it sends a CallFunctionRequest. The server code handling this request (ultimately leading to your function’s Run method) only receives the data present in CallFunctionRequest.
  • The server cannot compel Terraform Core to send the provider’s HCL configuration as part of a CallFunctionRequest, nor can it initiate an RPC back to Core to ask for it during a function call.
  1. Data Flow: The provider’s HCL configuration and the function’s arguments flow to the plugin through different, distinct RPCs at different times, as determined by Terraform Core’s execution plan.

What a Custom Server Could and Couldn’t Do:

  • Could Cache: If you were to customize the server, you could theoretically cache the req.Config DynamicValue (or the parsed resourcenamingtoolProviderModel) when the ValidateConfig RPC is handled.
  • Couldn’t Change RPC Payload: You couldn’t change the fundamental behavior that the CallFunctionRequest itself doesn’t contain this provider configuration.
  • Dependency on Lifecycle: Your custom server would still depend on Terraform Core actually calling ValidateConfig (or Configure) for the relevant provider instance before the function call, so that the data could be cached.

Your Current Approach: Your current method—using ValidateConfig to process the configuration and save it to a file, and using a data source to help ensure ValidateConfig is triggered—is a pragmatic workaround for the fact that the provider’s Configure method might not be called before a function invocation in “offline” scenarios like terraform validate. The function then reads this cached configuration.

Essentially, you’re already implementing a form of caching. The p.config] field in your provider, when populated by ValidateConfig or Configure, is an in-memory cache. If the NewGenerateResourceNameFunction constructor is called after p.config is populated by ValidateConfig, then f.config in your function instance would hold this data. The file-based sharing handles cases where p.config might not be directly available or consistent across different operations or if Configure isn’t run.

Modifying the gRPC server itself wouldn’t change the information Terraform Core sends with each specific type of request. The challenge lies in Terraform Core’s lifecycle for provider configuration in relation to function calls, especially in validate-only scenarios.

6

So basically implement the same validation for functions regarding the invocation of ValidateProviderConfig as with datasources?

I’d imagine this is less by design and more by coincidence that ValidateProviderConfig is occurring before ValidateDataSourceConfig. Terraform isn’t expecting providers to use this as a chance to store data in memory, that’s what ConfigureProvider is for :smiley: .

Can Terraform be modified to perform a ValidateProviderConfig RPC before a CallFunction RPC?

As mentioned before this isn’t really a technical problem, it’s design. So the question is, can we change Terraform’s design of provider functions to allow them to store data in memory from provider configuration? In which, if TF core wanted to do that, they should use ConfigureProvider to do so.

I’m not a member of the team that owns Terraform core (some peruse this forum, but I wouldn’t bank on them finding this discussion), so if you wanted to open a discussion about that, I’d go here → https://github.com/hashicorp/terraform/issues/new?template=feature_request.yml