I maintain a terraform provider for netbox, a typically self hosted product that unfortunately likes to do (small) breaking API changes. For example, the API changes the type of an object attribute from string to int.
Is there some kind of implementation pattern or example reference how I can make sure my provider works against multiple versions of the upstream API?
I feel like supporting multiple product versions will quickly become very tedious. On the other hand, only supporting one product version seems limiting, because all users of the provider would have to use the specific netbox version.
A few other providers have run into this. The strategies I’ve seen that I would suggest:
Establish an API support policy and matrix. “We support every version” seems untenable and a recipe for sadness.
Actively run acceptance tests against all the versions you support.
When you need different behavior from a resource for a specific API version, make an API version-specific version of that resource. For example, you could have compute_instance and compute_instance_v6. Sometimes you can get away with just versioning the field inside the resource–compute_instance.disk and compute_instance.disk_v6, for example–but be sure to think through the practitioner experience–e.g., is it ForceNew? Is that a bad user experience?–and test common use cases to make sure you don’t have perma-diffs or other problems.
Fundamentally, though, your Terraform provider is a consumer of the API, and so strategies from more traditional client libraries for the API may be applicable or offer inspiration for how you could handle things in your Terraform provider.
Another strategy is to stay in sync with the API, and to just publish breaking changes when the API breaks, letting people using older versions of the API use older versions of the provider, but that depends on the community around the API. If people tend to use older versions of the software or there’s a long tail of upgrades, that can create compatibility and support headaches. It can also be noisy and unpleasant for users if API breaking changes are too frequent.
Is there a pattern for building in compatibility checks to your provider - e.g. compare the API’s reported version against a list of “known good” versions - and messaging the end user if the two are incompatible?
And with version specific resources - that may be tricky (at least with the Netbox provider) as the underlying go library is the thing actually causing problems - this library is generated from the swagger spec and some errors bubble up when the API provides responses of which the library isn’t aware.
I’ve seen some providers go the route of tying the major release number to a specific API version (e.g. 1.x.x is for API version 10, 2.x.x is for API version 11) - thoughts on that option?
I’ve not seen it, but I can’t think of a solid reason why the ConfigureContextFunc couldn’t retrieve the API version of the deployment it’s pointed at and return a warning or error if it isn’t in a predefined range or why that would be a bad idea. Seems reasonable enough to me.
Does the underlying Go library not version its breaking changes correctly? If they do a major version bump, you could (in theory) just import both versions of the package. Not ideal, but it works. If they don’t version their packages appropriately… yeah, you’re kind of hamstrung by the upstream’s versioning hygiene.
I personally don’t like it, but Terraform doesn’t have an opinion on it one way or another. The reason I personally don’t like it is because you’re using the provider’s versioning schema to describe a different piece of software’s versioning, meaning you lose the ability to communicate the provider’s compatibility. In your example, if you need to make a breaking change in the provider within an API version, how would you do that?
There are ways around this, but it just makes things a little more challenging. It is a valid strategy to pursue, though, if it’s right for your situation and tradeoffs.