Plugin RPC API Documentation

Terraform Providers Plugin Development
1

Hi,

the documentation states that plugins are mostly written in go, and that terraform uses some kind of RPC api to communicate with the plugin, which i translate to, i can write a plugin in whatever language.

is there an documentation on this API? because i would like to write a plugin using python.

kind regards.

2

The documentation also says that no, the RPC API is not documented, and implementing in another language would be very challenging :slight_smile:

3

and i guess there are no active plans to document it?

4

I haven’t seen anything from HashiCorp indicating that, and based on what they have published I suspect it’s unlikely as it would increase their maintenance burden substantially.

5

Hey there,

Currently, our RPC for the SDK is using gRPC. If you look around in https://github.com/hashicorp/terraform a bit, you can find the protobuf files describing the protocol. There are also some docs on the protocol and resource lifecycles.

I’m being intentionally vague about this because right now the only support we offer is for building providers with our Go SDK. The technical ability for providers to be written in Python exists, but the support structures for it–places to ask questions, well-defined documents making it easy, communication channels tailored to developers operating at the protocol level, and tooling to support it–do not. Provider developers could, in theory, write a Python provider, but I would strongly encourage not doing that.

While we’re not closing the door on implementing providers in other languages, at the moment we’re focused on ensuring the developer experience with our Go SDK is up to our expectations. Other languages is a thing we’re keeping in the back of our minds, but it’s not something we’re actively investigating supporting at this time.

2 Likes
6

I think it’s just harder to manage installation of plugins written in non-compiled languages.

7

it is all about options.

providing a proper documentation would help externals devs to implement there own SDK i am currently trying to reverse engineer how the plugin API works, so i can implement a python based SDK. but it is a time consuming task :-/

8

Hey, how far along did you get with this idea? I’m running into a similar situation where I’d like the go-plugin library to properly support non-go plugins and may need to create my own SDK

1 Like
10

I would also be interested in introducing Terraform Plugins to the .NET world. But I think I have to go the hard way just like Paddy described by reading the source code, not very optimal.

EDIT:

What I can say so far, that this is pretty helpful as it describes how to implement the gRPC server:

Next it is important to understand how the MessagePack-encoded data structures are looking like:

Another thing to consider is to make your provider debuggable. Therefore you need to checkout:

Keep in mind that for windows the environment variable must look something like this (I didn’t find this resources on the same page):

{
    "registry.terraform.io/example_organization/example_provider": {
        "Protocol": "grpc",
        "Pid": 17940,
        "Test": true,
        "Addr": {
            "Network": "tcp",
            "String": "127.0.0.1:50727"
        }
    }
}

And last but not least it is helpful when you already have implemented a provider with GitHub - hashicorp/terraform-plugin-go: A low-level Go binding for the Terraform protocol for integrations to be built on top of. as it makes use of everything I just described above.

If you are interested in taking part into a C# implementation of a fully fledged Terraform Plugin SDK, please take a look at GitHub - pseudo-dynamic/PseudoDynamic.Terraform.Toolset: Implement Terraform plugins with best C# manners.

11

I’ve never used it myself, but have you seen GitHub - SamuelFisher/TerraformPluginDotNet: Write Terraform providers in C#. ?

12

Maybe this the wrong topic to write this down but the implementation you are talking about @maxb, is not yet finished sadly. It does not cover all kind of complexity terraform provides you to express objects including nested blocks, objects, list and and more. terraform-plugin-go uses go-cty as example and so I do with introducing a proper type system: therefore I translate first C# classes to my type system and then to MessagePack representation and back, similiar to terraform-plugin-framework/types or terraform-plugin-go/tftypes.

The implementation you are talking about skips this step so the mapping class need to be written in view of MessagePack. Therefore the mapping classes need first to comply with MessagePack conventions and then additionally with Terraform MessagePack data format conventions which is tough. Another story are unknown values which are currently not representable by this implementation.

So I can only repeat myself: If you are interested in taking part into a C# implementation of a fully fledged Terraform Plugin SDK, please take a look at GitHub - pseudo-dynamic/PseudoDynamic.Terraform.Toolset: Implement Terraform plugins with best C# manners. It still needs development time, so any help is appreciated. :smiley: