Because state file may have some sensitive information like password, which may cause security problem. I can’t execute “terraform apply/plan” command as a dependent process in my host.
so I want to develop a GRPC client to communicate with plugins, so that I can guarantee all the operation and information only in memory.Does anyone have experience on this？
If I only need to call plugin interface to CURD resource(no need to compare the differents), how much work？
Hi @XiangyuQin,
What you’re thinking about is possible in principle, but does involve some challenges you’ll need to think about how to overcome:
-
There are three major versions of the provider protocol in use across all historical provider releases. Protocol-4-only providers don’t work in Terraform v0.12 or later so you can probably ignore those if you stick with recent provider releases, but in the current set of stable releases at the time of writing you’ll find a mixture of protocol versions 5 and 6, both of which have separate, incompatible gRPC schemas. (Breaking changes to the gRPC schema is what a new major version represents.)
For your purposes then you’ll may need to implement a client for both protocol versions, if the plugins you intend to use don’t all agree on which they support. We have no plans to specify another major version 7 in the foreseeable future, but that might come eventually and so at that point you’d need to either keep using older provider versions or implement support for the new protocol in your client.
-
Terraform CLI’s provider installer is not separable from the other work
terraform initdoes, so you’ll need to either manually fetch and extract the specific plugins you want to use or design your own client for the provider registry protocol if you want to achieve automatic installation. -
The provider protocol is designed around the idea of planning and then applying changes, and not around “CRUD”, so you’ll need to translate your CRUD operations into equivalent requests to create a plan and then apply the resulting plan.
For example, if you want to implement “create” then you’d request a plan with the old object set to null and the new object describing what you want to create. For “delete” you’d invert that, so that the old object is what you created previously and the new object is null. To plan an update, you need to send both the object which came from the most recent change and the new settings you want to apply. For some proposed updates, the provider may inform you that it can’t update-in-place for that change, and so you’ll need to implement it instead as a destroy and a create, as Terraform would.
Notice that this means you’ll need to retain the previously-created objects somewhere, even if you only pass it verbatim back to the provider for later changes, mimicking what Terraform does with state snapshots, because providers always plan changes in terms of the previous object. Given that your goal was to keep changes in memory only, this particular challenge might defeat your whole idea. However, if you can adjust your plan to instead persist the data securely at rest, or if you constrain your goals only to creating and not to updating or deleting then it may still be feasible.
-
As a more general caution: there’s a lot more special semantics and constraints on the behavior of a client and server of the Terraform plugin protocols than just the structural constraints enforced by the schema. Provider plugins generally assume that their client will behave like Terraform behaves (really, they assume their client is Terraform) and so if you don’t match Terraform’s client behaviors closely – for example, if you ask a provider to apply an update change that it already told you it cannot apply – the providers may behave in unpredictable ways, including possibly crashing.
A long time ago I wrote a prototype of calling directly into the providers like you’re describing, which you might find useful as a starting point. This was a spare-time experiment, not something I intend to keep maintaining, and not an official HashiCorp project, so I would not suggest depending on it as a library, but you can feel free to study it to see how I implemented the gRPC setup, etc. Note though that because that experiment predates protocol version 6 it supports only protocol version 5 currently; it will take some additional work to make it also support protocol version 6 and then negotiate with the provider plugin to dynamically choose a suitable protocol version.
Thank you very much!! this is great help for us. it looks like I still have a lots of job to do(including version iteration adaptation), I think I need to discuss this plan further with my team