I think there should be a GRPC datasource that behaves like the http datasource: Terraform Registry
Being able to build a GRPC call which is often non-trivial and requires proto files, etc in terraform would be extremely valuable.
It could be used to fetch data as the http datasource does but it could also be useful for documentation and testing.
You could even use the http datasource to pull proto files down from github do use in the GRPC datasource call!
A lot of the GRPC tooling is clunky and annoying to use (grpcurl) and being able to describe a GRPC call declaratively in TF would be really helpful.
Anyone can write a provider and publish it in the Terraform Registry, so I think the main blocker for this is more about designing exactly what it would do before implementing it.
Unlike plain HTTP, gRPC relies on type information in the form of protocol buffers descriptors. You didn’t mention what makes grpcurl “clunky and annoying to use” for you, but one reason it’s less ergonomic than curl is that it needs to get the descriptors from somewhere, and that would also be true for a hypothetical provider offering similar functionality.
So one big category of questions that someone designing a provider for this would need to answer includes:
-
Would the provider need to support the same three different ways to obtain descriptors that grpcurl offers, or is it acceptable to e.g. make the provider only work with servers that implement the “service reflection” protocol?
-
If supporting anything other than automatic discovery using service reflection, are the descriptors configured as provider-level arguments or as data-resource-level arguments?
The first option would make it more convenient to write many data blocks referring to different parts of the same service API, but the second option would make it more straightforward for one-off use with a single data block, so the designer would need to decide which of those two situations to prioritize.
-
If using the service reflection protocol, is it okay to re-fetch the descriptors for each call or should the provider attempt to cache the descriptors in its own RAM? Or even try to cache them in some temporary cache location on disk?
To be clear I don’t intend this to mean that it’s a bad idea for someone to write such a provider. I’m just saying that the fact that gRPC is more “clunky” to use than HTTP is at least partially caused by differences in gRPC’s design, and so some amount of that clunkiness would also transfer to a Terraform provider offering similar functionality. I still think it would be quite neat to have such a provider, but I can understand why nobody tried to write it yet.
Im just thinking a simple grpc equivalent of the http datasource- definitely not a separate provider.
Perhaps native, but grpcurl is golang and provides a library package specifically for supporting other tools, I was expecting this would just be a thin wrapper around that. I’m not sure how grpcurl does caching though.
Hi @red8888,
The http data source belongs to the hashicorp/http provider. All data sources belong to providers.
A hypothetical grpc data source should belong to a provider called “grpc”, because Terraform assumes that the first segment of the name of the resource type is the name of the provider it belongs to, but that provider could be in any namespace. So you can write such a provider and publish under your own GitHub account in Terraform Registry, and then Terraform will be able to install and use it.
Looking at the grpcurl Go API, it seems like your hypothetical data source would mainly wrap grpcurl.InvokeRPC, but that requires a DescriptorSource argument, and so a different way to think about the questions I asked in my previous comment is “How would the DescriptorSource be specified in the Terraform configuration?”. Using this library doesn’t avoid answering that question, unfortunately.