Pygo-plugin: Python host support for hashicorp/go-plugin

Terraform
1

Hi Everyone,
I am posting this to the Terraform category since go-plugin does not have its own, and I see one other similar kind of post has also been made here to cover that project.

I recently finished prototyping within my company to see if hashicorp/go-plugin could make sense being ported for Python host (client) support. My approach was to create a binding against the Go client so as to reuse all of the existing Go logic, and then implement the plugin (server) specialization code in pure python. I ended up posting it as a public project in its alpha state, and do intend to do more work on it at some point when my time frees up again. But I also thought I would mention it here in case anyone finds it interesting and wants to throw some contributions at it to see it fully implemented?

As I mentioned, it currently uses a binding approach via gopy, and from this work I ended up submitting a number of gopy updates. The downside is that gopy seems to be a bit broken with windows build support at the moment, but it could be fixed.

The basic features are there and the examples work. Hope the community finds this interesting as a way to use the same plugins from a go-python setup in a Python-based host.

2

Thanks for sharing this, @justinfx!

You might find my earlier side project (not a HashiCorp project) rpcplugin interesting in similar vein. I was attempting to define a go-plugin-like (and largely compatible) protocol without all of the historical baggage of supporting the Go-specific net/rpc transport, and with a spec along with the implementations.

I’m not actively working on it at the moment because I have lots of other things going on in my life :smiley: but you might find the python implementation interesting for your goals.

3

Oh cool. Guess that might have been a nice starting point if I knew it already existed :slight_smile:
I also just ignored the net/rpc implementation in Python and it didn’t seem like much of a big deal since its just a named protocol amongst the two supported ones. But it is definitely simpler from a deploy standpoint having it be pure python instead of bindings. I didn’t need to implement too much on the client side though since I was able to just sit on the shoulder of whatever the go-plugin client implements. Most of the custom stuff would be in the plugin side to get it to do all the nicer extras like the fancy log stream handling and tls.

Thanks for the reply!