Terraform gRPC alternative client implementation

Hi @mll,

It is possible in principle to write a “plugin client” (which is what we call the process that launches the plugins) in a language other than Go, but there are more parts to implement than just the gRPC client layer. For example:

  • gRPC only deals with the messages sent between client and server once the server is running. Terraform also has a specific protocol for launching the plugin process and negotiating a communication channel with it, which includes a TLS key exchange and selecting a network channel to communicate over.
  • The Terraform plugin protocol is defined in terms of gRPC, but there are higher-level behaviors you will also need to match in your own code in order to successfully interact with a provider. For example, you will need to be able to encode dynamically-specified values such as the resource configuration in the way that Terraform providers expect, which is just a byte array at the gRPC layer but of course that array must contain data in a suitable format that is not visible in the protobuf schema alone.
  • Terraform also includes functionality for interacting with a provider registry to automatically install providers, which may or may not be important to your particular goal.

Terraform currently interacts with its plugins using a Go-specific library that has no associated protocol specification. You’ll have the easiest time if you write your software also in Go and use go-plugin to implement the plugin handshake.

However, unofficially (which is to say: this is not a guarantee made by the Terraform team to remain true for all future protocol versions) the protocol is more-or-less compatible with the derived RPCPlugin Protocol that isn’t a HashiCorp project, and so with some care to review the appendix about HashiCorp go-plugin compatibility I have successfully written side-projects that interact with currently-published Terraform providers using implementations of the rpcplugin spec, and so it may be a useful starting point for implementations in other languages.

I will caution that the API offered by individual providers is lower-level than you might hope, and that the runtime engine in Terraform Core (built around HCL) is a large part of what makes the Terraform workflow work. It is of course possible to implement an alternative runtime around the same providers, but you should not expect that reusing the existing Terraform providers will make the implementation work trivial on your side.