Should I pin my terraform providers?

Terraform
1

Hi,

My goal is to understand how important dependency pinning really is.

I am aware that dependency pinning is a good practice and I should pin the version of my terraform providers.

That being said, I’m working with teams that have never used terraform in the past. They have a lot of other priorities to deal with.

I’m interested by feedbacks from the terraform community : have you already used terraform without caring about provider dependency pinning? If so, what has been the most painful issues you had to deal with? How frequently? Once per year?

Note: I’m not talking about theoretical issues that can happen but real issues that have happened to you or your colleagues.

2

So, here is a real world example:

An internally shared module was created that provided a method of deploying groups of PaaS resources in various standard configurations. Provider version constraint was a simple >

This shared module was used quite widely by various teams, developers and projects, with no issues for some time.

The main provider used for the module did some breaking changes (prior to this they had ensured there were deprecation notices - which everyone ignored as their deployments continued to work).

  • Older, stable, projects using the module were unaffected. They had their .terraform.lock.hcl checked in (as recommended) and continued to use the provider version that was part of that lock file (Which was committed prior to the breaking changes).
  • Some newer more ‘dynamic’ projects suddenly stopped working after updates (where the providers had been upgraded to get access to updated resources) but others of the same type continued (where they were still seeing change but the provider was not updated).
  • Some Devs also saw issues on their local projects, but others didn’t - again, dependent upon their root module’s provider constraints and their lock file contents.

As this occurred over a broad range of projects that were using that module and at different points in time over a week or two it was not the most straightforward to fix (even if the fix was simple) but instigated a code review policy that now mandates that for provider version constraints they :

  • Must include a constraint on the minimum version.
  • Must include a constraint on the maximum major version.
  • Should use the ~> #.# or the >= #.#.#, < #.#.# format.

Since then we have had no issues.

But, as illustrated, the predictability and impact of such issues is difficult to quantify and better to have to explicitly change something (e.g… update a version constraint, run a terraform init -upgrade and then commit that code via review) than to have failures triggered by a change of provider version at an arbitrary time. Especially when working with less experienced terraform users for who the interactions between module and sub-module constraints and versions, the lock file, etc. may not be clear and that they may find difficult to troubleshoot.

I would much rather have a junior come to me with a “I can’t get this to work” related to accessing a feature from a newer version that is being prevented by constraints, constraint combinations and lock files, than come to me with an “it’s all stopped working and people are shouting and I don’t know why” :slight_smile:

Just my view, others may differ :smiley:

1 Like
3

Have you ever used Terraform without pinning provider dependencies?

4

When talking about providers in particular (not about modules or Terraform Core’s own version), “dependency pinning” in the sense of specifying a single exact version of each provider as part of your root module is no longer needed since Terraform v0.13, because Terraform uses a dependency lock file to automatically “remember” which version of each provider has been selected.

These features are designed to be used in the following way:

  • Each of your modules can optionally specify a minimum version that you know it’s definitely compatible with, using a >= version constraint in the required_providers block.

    For example, if you know your module is currently compatible with hashicorp/aws v5.55.0 then you could declare that like this:

    terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.55.0"
    }
  }
}

    This ensures that terraform init can raise an error if someone tries to use this module as part of a Terraform configuration that has already selected an earlier version of this provider.

  • After you’ve run terraform init, include the generated file .terraform.lock.hcl in your version control system to record which specific versions were selected.

    Terraform will then guarantee to select exactly those versions in future unless you explicitly ask it to upgrade using terraform init -upgrade. After upgrading you can review the proposed changes to the dependency lock file to confirm whether the version selections match with your intentions.

  • If a provider you are using publishes a new major release with breaking changes, the dependency lock file means that you will remain on the last-known-good version until you’re ready to try upgrading, at which point you can use the -upgrade option described in the previous point to indicate that.

    If you try upgrading and find that the breaking changes have broken your module then you have two main options for how to proceed:

    • If the fix is straightforward and you want to adopt it immediately then you can make the changes you need to make and update the module’s version constraint to select the new major version of the provider.

      (You might also choose to publish a new major version of your own module, if you expect that requiring the new major version will be disruptive to configurations that use other modules that haven’t been upgraded yet. Users of your module can then remain on the previous version until all of their dependencies are ready to use the new major version.)

    • If the fix is too complex to embark on immediately and thus you expect the module to remain in a “broken with latest version” state for some time, you can optionally publish a new version of the module with an upper bound on its version constraint, making it explicit that the new version of the provider isn’t compatible:

        version = ">= 5.55.0, < 6.0.0"

      This will then avoid new users of your module selecting a provider version that isn’t compatible with the module, and in particular will block using your module in conjunction with any other module that requires the new major version of the provider.

      I don’t recommend proactively adding upper bounds to version constraints until you actually know your module is broken with a newer version. While it can be tempting to be conservative, a new major version may or may not break your module’s behavior and being too conservative in proactively blocking newer versions will make it hard to use your module in conjunction with others that have different version constraints. I think it’s better to be optimistic about future version compatibility until you know for certain that there’s a problem.

The advantage of using dependency locking instead of dependency “pinning” is that in the happy (and hopefully common) case – where a new version is backwards-compatible – you can just run terraform init -upgrade for each root module, send the updated lock file through code review, and move on. The equivalent with dependency pinning requires visiting each module and manually updating its version constraints, which is more laborious and introduces more risk of different modules ending up disagreeing with each other about which modules they are compatible with, creating dependency hell.

Current Terraform does not support automatic dependency locking for shared modules, so “dependency pinning” (specifying a single exact version to use) remains the best compromise if you prioritize reproducibility – though not all teams do prioritize that! – but there’s less chance of “dependency hell” for modules because each module block is allowed to depend on a different version of a shared module, whereas for providers Terraform requires that all modules in the configuration must agree on a single version of each provider that they are compatible with.

5

Dependency pinning in Terraform is crucial for stability, as not pinning versions can lead to real issues like breaking changes in providers, inconsistent environments, and increased debugging time. These problems can occur unpredictably, making it essential to pin versions from the start, even for teams new to Terraform, to avoid potential headaches and ensure consistent infrastructure.

6

Yes, pinning your Terraform providers ensures consistency and stability by locking them to specific versions. This avoids unexpected changes and compatibility issues during deployments. enjoy the best app link hear:honista apk app

7

This topic was automatically closed 62 days after the last reply. New replies are no longer allowed.