I want to write a provider which has a resource A. To create an A instance, terraform need to invoke our service API in resource A’s Create method. When creating multi instances using count or foreach, it will invoke API multi times (maybe). But when there are many instances need to be created, these invocation will be a burden to our service. So I’m curious if there is a way to invoke a batch creation API to create multiple instances at once, instead of calling the API every time to create a separate instance.
Adding to that, I may be concerned about how to implement the Create interface for a resource and if there are any additional considerations to take into account when designing and using states if our service offers a batch creation API
The interface between Terraform core and a Terraform provider is defined via a protobuf schema:
No batch operations exist.
That means there’s no good way to do batch API requests in Terraform.
There is a possible bad way, but it’s limited, and messy: you could have a provider delay incoming requests to work on resources for a short time, to see if Terraform core concurrently asks the provider to do similar work on multiple instances of a resource type.
However, Terraform core deliberately limits the concurrency of provider calls, so the provider will not have all the information to make an optimal number of batched requests.
The provider cannot be sure whether Terraform core is now blocked waiting for some of the calls to it to complete, so it would have to time out its internal delays, and act on whatever batch size had so far been provided to it.
This all seems messy enough that it’s probably not a worthwhile option to engineer.
