My API spec is like this :
Basic CRUD, then advanced CRUD like multiple create at once, multiple delete at once.
Can this feature be developed on the provider side ?
Hi @ramuklawjju,
Terraform’s model doesn’t include any explicit support for combining multiple operations together into a single provider call, so if creating multiple objects together in a single call is an important part of the user model (as opposed to just an optimization) then the typical answer is to define a resource type with a plural name that explicitly creates multiple objects at once, failing as a whole if any one of them can’t be created:
resource "example_things" "example" {
thing {
# settings for one thing
}
thing {
# settings for a second thing
}
}
Since your question is very general I can’t really be any more specific than that, but if you have some follow-on questions and can share some more detail I’d be happy to elaborate more.
Hi @apparentlymart ,
As part of my requirement, I’ve two DELETE api calls, one which takes id as a parameter to remove a server, another takes zone as a parameter and removes all the servers from that zone.
My concern is, how to implement the last delete call which deletes all servers by zone using terraform?
It doesn’t sound like you would need to implement the delete all servers call as presumably you would have a resource that represents a server. Therefore you’d only be acting on one server at a time to add/edit/delete. If Terraform ascertains that several servers need removing that happen to be in the same zone it would just cause the plugin to call the /server/{id} DELETE call several times.
Hi @stuart-c ,
In my post call, zone is a request parameter, so user creates multiple servers in different zones and now he wants a configuration to remove all the servers from a particular zone.
Not able to remove them will be a bad experience since the api already supports it. So I was looking a way to incorporate changes so that my provider closely represents what the APIs are doing.
My question here would be, if there is a way to make my provider do it ? I can add a parameter say delete_all_flag if necessary, is it possible to achieve this ?
With Terraform it is all about the resources you are managing, with all changes being made to those resources. Nobody sees anything about the underlying API calls that might be being used, so there isn’t a huge difference between a single call that removes all servers in a zone and several calls which remove all the servers by ID - from the user’s perspective you are just removing a few server resources.
This question is similar to another recent one about APIs which have batch calls available (batch get, update, create, etc.). As Terraform works on each resource one by one such calls aren’t needed/useful. The only impact would be changes possibly being slightly slower, although you also get the advantage of parallel changes and the ability for partial failures.
Since cost is a factor when resources are being discussed and it does matter in my case if it is a one by one or all at once.
Also for partial failures, it’ll be a bad experience on the user side if they have to take care of it rather than expecting the api to do it. I’m aware Terraform works on the resources one by one, was looking if there is a workaround for this requirement ?
Also overall experience would get impacted, if user adds servers using terraform but have to switch to the API for removing servers by zone.
If possible can you share the link to the similar question about the batch calls ?
The user wouldn’t have to do anything special to delete the servers or handle failures.
For example suppose a user writes the following code for your provider:
resource "server" "server_1" {
zone = "a"
}
resource "server" "server_2" {
zone - "a"
}
When initially run it will cause your POST /server call to be used twice returning two new IDs. If they then removed both resources and ran terraform apply again it would realise it has two servers it no longer wants and therefore cause the DELETE /server/{id} request to be called twice, ones for each of the two IDs previously generated (stored in the Terraform state file). If you assume that one of those calls failed for some reason, the state file would only be updated to remove the one which succeeded. Running terraform apply again would then see the second server still needs deleting so would trigger the DELETE /server/{id} call again (maybe passing this time).
The user doesn’t have to understand anything about the APIs. They just have to think about which server resources they want and with what settings. Terraform will handle adding/removing/updating servers as needed using whichever API call to ensure reality mirrors what the code says. While in the example above the step that removes both servers would be slightly more efficient in the success case using the DELETE /server/{zone} call (1 API call instead of 2) if it were to fail you’d end up in a possibly bad situation - either Terraform state wouldn’t change [so Terraform things both servers still exist even if one no longer does] or even worse it gets updated and there is now a server in existence that is no longer being managed. Even if there is a cost per API call I’d expect it to be so small as to make no practical difference between 1 or 2 calls (with something like AWS most API calls are small fractions of a cent each).
Hi @stuart-c
Suppose the account has dev/test/integ/dr zones where multiple users spin up some servers, like in a shared account with multiple users and now the testing is over, and they want to remove the servers.
When the users initially provisions, they have created multiple configuration files for different features and different resources and almost all of our features have batch removal, so basically it works like once batch delete is hit, it triggers removal of resources from the account, so from user side hitting the batch api is needed. That’s why I’m looking a way to incorporate it with my provider.
Any suggestions on how I can achieve this ?
Calls to the batch APIs aren’t needed/possible.
If a user was to delete all servers in a zone (either via a batch API call or using some sort of web interface) the next time you run terraform apply it would notice the missing servers and create them again.
As a result if you are managing a set of resources via Terraform you should not update or remove them using any other mechanism than by updating the Terraform code in your .tf files.
How a user might structure their code is up to them, but a common pattern when wanting multiple identical setups (multiple envirnoments) is to create a module describing the blueprint and then instantiate it multiple times. If they then decide a server isn’t needed any more they would update the module, run Terraform and the server would be removed from each environment. If a whole environment isn’t needed they would remove the module call and rerun Terraform, causing all servers to be destroyed.
Terraform handles all this for the user. At no point are they needing to understand the underlying API calls. Your provider is informed when a specific resource needs creating, updating or removing to allow you to send those API requests as needed. Each resource is independently managed so you wouldn’t be aware/care about multiple servers being created/updated/removed at once.
if a user does terraform apply or plan and the server is deleted, it will show in the plan that a new server will be added, right ? So that’s not a concern.
As @stuart-c has said, in Terraform’s model users are typically thinking about resources rather than specific API calls, and so as a provider developer our job is to figure out how to map CRUD on individual resources to the remote system’s model.
Part of that design job, then, is to decide how to map the remote system concepts onto Terraform’s concepts. The most common answer is to define one resource type per distinct object in the remote system’s domain model, in which case each individual object would have its own independent lifecycle and you would only ever be using your single-object CRUD operations to implement those independent lifecycles.
An alternative design would be for you to decide that “zone” is a resource type but that a “server” is considered just as a part of a zone, rather than a distinct object with its own lifecycle. In that case, the configuration might look like this:
resource "yoursystem_zone" "example" {
name = "a"
# (other zone-related arguments)
server {
# (server-related arguments)
}
server {
# (server-related arguments)
}
}
With this design, your implementation of yoursystem_zone would presumably implement “create” by calling POST /zone followed by zero or more POST /server, but it could implement “delete” by calling DELETE /server/{zone}.
The most important advice I’d give here, details aside, is that a Terraform provider should typically be designed around the user model of the system – that is, the set of objects the user is familiar with from a management UI or from documentation – rather than around the physical API of the system, which is just an implementation detail. Whether your system presents servers as part of zones or as distinct objects from zones is the much more important question for your provider design than what API endpoints you have available to implement those operations.
1 Like