Hey Terraform Provider Developers!
Just as an update on this topic, we got feedback that getting a bit more information would be helpful in preparing for upgrading to version 2.0.0 of the SDK. Specifically, people are trying to gauge how disruptive the breaking changes we intend to make are, and how much time they should allocate for the upgrade process, along with what new features we’re introducing.
Because of the many layers of compatibility shims and the variation in the usage of the SDK, we sometimes need to modify, postpone, or cancel changes to the SDK, even late in their development cycles. We want to caution that this list is, by nature, incomplete and subject to change. It is the latest information we are working with, but some of these changes are still being made, and remain subject to more thorough testing. We hope it will help you understand the scope of version 2.0.0, and will help you plan, but you’ll forgive us if the final release does not match this list exactly.
Breaking Changes
There are a handful of breaking changes that we anticipate will affect only a small percentage of developers, mostly revolving around built-in helper functions. To keep this post reasonably-sized, we’re going to limit ourselves to changes we believe will affect the majority of developers.
Dropping Support For Terraform 0.11
The largest breaking change providers will need to be aware of is that v2.0.0 of the SDK will not support Terraform 0.11 or before. Terraform 0.12 or higher must be used with providers built against versions of the SDK after v2.0.0. Our upgrade guide will include information about how to make sure Terraform 0.11 and below do not accidentally download these versions of the providers when running terraform init.
0.12 introduced a large number of new ideas and features to Terraform, and thus far the SDK has only adopted those that could live side-by-side with 0.11. Unfortunately, a large number of improvements we would like to make to the SDK cannot be made while maintaining support for 0.11 and below. We’ve decided to focus on bringing more features to the SDK, instead of maintaining compatibility with 0.11 and before.
If your provider has not already ended support for Terraform 0.11, HashiCorp recommends you ship a new major version of your provider when dropping support.
See our blog post for more background on this change.
Removing Partial State
The ResourceData.SetPartial and ResourceData.Partial methods in the helper/schema packages were poorly understood and usually unnecessary. Most developers will just need to remove them from their code; in incredibly rare cases, a developer may need to update a resource’s design. Guidance on when either method was actually necessary will be included in our upgrade guide.
Panic On ResourceData.Set Errors
Previously, ResourceData.Set would return an error when trying to set data that doesn’t fit in the schema for the specified field. These issues were almost entirely programming errors, not usage errors. As such, these errors were routinely not checked, and would lead to unfortunate bugs from developer typos or type errors. We introduced a TF_SCHEMA_PANIC_ON_ERROR environment variable years ago that, when set, would cause the SDK to panic instead of returning an error. This was widely used in CI and helped prevent a number of bugs from making it to releases. With this experience, we’re now confident this is the correct default behavior for the SDK, and have updated it accordingly.
Now when ResourceData.Set would have returned an error, it will panic instead, unless TF_SCHEMA_PANIC_ON_ERROR is explicitly set by the user to a falsey value like 0 or false.
Deprecate Exists
While current code won’t break, we want to draw developers’ attention to the Exists function for schema.Resources. It is now deprecated, and we recommend moving its logic into your Read functions, instead. We found most of this logic got duplicated and run twice, anyways, for no gain. We’ll include instructions in our upgrade guide on how to move Exist functions into Read functions with minimal effort.
Context-Aware Functions
Under v2.0.0 of the SDK, the SDK accepts context.Contexts in a lot more functions and methods, which enables easier usage of defined timeouts and cancellation in providers. Most functions–like the Create, Read, Update, and Destroy functions that are the backbone of providers–are merely deprecated, with context-aware variants being available. Upgrading to the new context-aware versions of these functions should require only changing the name of the property the function is set on, and adding a context.Context to the parameters for the function. But they also do not need to happen immediately; the deprecated versions will still exist and work, we just recommend migrating off them as soon as it’s convenient, as they’re not guaranteed to stick around forever.
Some helper functions–most notably, CustomizeDiff–do not and cannot have this backwards compatibility available to them. They will require the context.Context to be added to their type signature when upgrading.
A Cleaner API Surface Area
A great many functionally internal methods, types, variables, and packages were removed from our public API–either deleted outright, unexported, or moved into an unimportable internal/ package. These parts of the API surface were never meant for consumption by provider developers, and we took the opportunity of a breaking change to remove them from the API, helping to enforce that expectation and more clearly communicate it.
New Features
With these breaking changes, we’re bringing a number of quality of life improvements to the SDK. This post is too long already, so we won’t cover all of them, but we want to talk about some of the main things we think people are looking forward to.
Diagnostics Support
The opt-in context-aware versions of the Create, Read, Update, and Destroy functions that providers rely on return a new Diagnostics type instead of an error. While using this new feature will require a small amount of work–we expect much of it to be search and replace, and will have more information in the upgrade guide–it enables a long-standing request and improves end user UX considerably. First, providers can now return warnings from Create, Read, Update, and Destroy functions. Second, errors (and now warnings!) that a provider generates will be more narrowly targeted at the exact field, sometimes even the exact item in a field, that prompted that feedback.
This is also automatically applied to validations using ValidateFunc in providers that upgrade, giving more precise code locations of validation issues.
More Accurate Test Runs
Previously, our test framework had a mocked copy of Terraform’s core logic that would simulate a Terraform run. As with all simulations, it was an imperfect-but-passable approximation of what Terraform actually did. In v2.0.0 of the SDK, we’ve modified the test framework so now tests run an actual Terraform binary behind the scenes. This ensures that the behavior you see in test runs matches the behavior your users will see in production, and it also opens up exciting possibilities for running tests against various versions of Terraform. Testing is obviously a crucial piece of Terraform provider development, and something we want to be careful about, so we’re still finalising details on this, but we anticipate it being largely a drop-in replacement for existing testing code. Notably, developers that manually check the values of items in a set in their tests may need to change those specific checks.
Feedback Welcome
These are the major features and breaking changes we anticipate making that we expect to have the most impact on provider developers. It is not exhaustive, and it’s subject to change, but it’s the latest information we’re working with. If you’d like to get a more detailed picture of what this all looks like for your provider specifically, we recommend running a test run of your provider after changing your SDK dependency to point to the master branch.
We’re always happy to receive feedback, and welcome any bug reports. Feedback about the process or that doesn’t have an easily defined “done” state it would be awesome to centralize here on the forums; bug reports are encouraged, as always, in the issue tracker.
Happy Terraforming!