Specify multiple `version` strings within `required_providers`?

Terraform
1

I’m surprised to find that this configuration is permitted:

terraform {
  required_providers {
    aws = {
      // the last version listed seems to win -- 5.97.0 will be installed
      version = ">5.99.0, <5.100.0" // allows 5.99.1
      version = ">5.96.0, <5.98.0"  // allows 5.97.0
    }
  }
}

As far as I can tell, this configuration is valid (Terraform 1.11.4), but I can’t tell what effect it has. So far, it looks to me like the last version constraint is the winner.

Is there some utility associated with allowing multiple version attributes for a single provider local name?

I got here because I was surprised by the output of terraform-config-inspect when passing it a version string containing two version constraints, e.g. version = ">5.99.0, <5.100.0".

I expected JSON output like this:

{
  "aws": {
    "version_constraints": [
      ">5.99.0",
      "<5.100.0"
    ]
  }
}

But I actually got this:

{
  "aws": {
    "version_constraints": [
      ">5.99.0, <5.100.0"
    ]
  }
}

The string >5.99.0, <5.100.0 unpacks to a version.Constraints, which is actually a []*version.Constraint.

A JSON array of strings, each representing a []*version.Constraint means we’ve got a two dimensional array of *version.Constraint types.

Is this the intended behavior?

2

That looks to me to be a mistake by the terraform-config-inspect library, though perhaps the actual behavior is undefined. You can’t logically assign multiple values to the same attribute, so only one constraint string will be seen, and from the order of operations when parsing the language the final assignment is the one which will take effect.

The CLI seems to reflect that assumption, and terraform providers will show only the constraint from the final version line.

3

That was my first impression also.

I think you’re saying that my required_providers stanza above is nonsense. Do I understand that correctly?

Assuming there’s no utility in specifying version twice as I have done, then appending to the version constraints slice, doesn’t make a ton of sense.

I patched terraform-config-inspect to behave the way I expected. In fact, it was that work which led me to try the silly configuration I offered above.

Do you think that a PR which separates the constraints in the way we both seem to have expected would be welcome? It uses the same “final assignment takes effect” approach which you have described.

Long story short: I want a version of terraform-config-inspect which parses the version constraints sensibly, and I’d rather submit a PR than maintain a fork.

4

yeah, looking at this more, the terraform-config-inspect version was more meant to reflect what is in the config, not necessarily it’s meaning. The constraints aren’t parsed, we just collect the strings and present them in a json format.

I was thrown off a bit, because your example output doesn’t match the actual output from terraform-config-inspect, which would be

    "aws": {
      "version_constraints": [
        ">5.99.0, <5.100.0",
        ">5.96.0, <5.98.0"
      ]
    }

reflecting the exact strings from the configuration. Appending it was a choice to allow some tooling to be able to ingest broken or invalid configuration, and is not representative of the Terraform CLI behavior.

The CLI then outputs

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/aws] > 5.96.0, < 5.98.0

Yes, the configuration is a bit nonsensical, but because of Terraform’s object handling style which allows for various overrides and partial definitions, the multiple assignment was let through.

5

It sounds like you’ve concluded that the terraform-config-inspect behavior is correct because it’s showing what’s in the configuration (even when what’s in the configuration is nonsense).

That was the gist of the question. If the output is correct/expected, then I need to adjust my expectations.

Regarding your “actual output” comment, yes, that’s the output you’d get from terraform-config-inspect-ing the example configuration.

I confused things by explaining how I got here, using the single line, sane configuration I quoted directly above the terraform-config-inspect output samples.

6

Yup, that’s correct. I thought you were reporting that it would only show the first of multiple entries, which wouldn’t match either the intended behavior, or the effective behavior of the Terraform CLI.

7

Hi @jbardin,

I’m not sure that the “reflect the actual configuration contents” explanation makes sense.

Consider this nonsensical configuration:

terraform {
  required_providers {
    aws = {
      source = "bogus/aws"
      source = "hashicorp/aws"
      version = ">= 6.5.0, < 6.6.0"
      version = "~> 6.5.0"
      configuration_aliases = [aws.foo, aws.bar]
      configuration_aliases = [aws.baz]
    }
  }
}

The relevant terraform-config-inspect output is:

{
  "aws": {
    "source": "hashicorp/aws",
    "version_constraints": [
      ">= 6.5.0, < 6.6.0",
      "~> 6.5.0"
    ],
    "aliases": [
      {
        "name": "aws",
        "alias": "foo"
      },
      {
        "name": "aws",
        "alias": "bar"
      },
      {
        "name": "aws",
        "alias": "baz"
      }
    ]
  }
}

We have several contradictions to reconcile:

  • The output includes only a single source attribute, when two were specified in the configuration.
  • The configuration’s version attribute has been renamed to version_constraints, suggesting that some transformation might be in play
  • The configuration’s configuration_aliases attribute has been renamed to aliases, suggesting that some transformation might be in play (and it is)
  • The configuration specified two collections for configuration_aliases, but these have been flattened into a single JSON array.
  • The each element within the configuration’s version_aliases collection has been parsed from a simple string into a new object, so parsing and transformation is clearly allowed/expected.

Long story short, it really looks to me like the version attribute (a string representing a collection of constraints) should have been parsed into individual elements of the version_constraints array, similar to how the aliases are handled.

Side note, I suspect the alias handling should have overwritten the first value, rather than flattening values from multiple contradictory lines into a single collection.

8

Yup, it looks like the two implementations there don’t agree. In practice that library is made to expose only a limited portion of the config for some specific consumers, and while it may be overly lenient in some cases, it usually only sees config which has already been validated. Whatever is consuming that library’s output probably also accounts for any discrepancies (that PR was submitted on behalf of the language server, but I don’t know if it’s still used in that capacity at all).

  • The output includes only a single source attribute, when two were specified in the configuration.

That would be consistent with the final assignment “winning” when parsing a language like this.

  • The configuration’s version attribute has been renamed to version_constraints, suggesting that some transformation might be in play

There must be a conversion, because this is being parsed from HCL into some custom json structure. The naming changes are mainly due to historical changes in the configuration, but the team building and maintaining this tool wanting to keep an existing name for compatibility.

  • The configuration’s configuration_aliases attribute has been renamed to aliases, suggesting that some transformation might be in play (and it is)

Same thing, a different tool deciding to keep the historical name for their own use.

  • The configuration specified two collections for configuration_aliases, but these have been flattened into a single JSON array.
  • The each element within the configuration’s version_aliases collection has been parsed from a simple string into a new object, so parsing and transformation is clearly allowed/expected.

The only specified (albeit mostly by implementation) behavior here is what the actual Terraform CLI does when faced with the configuration, so we can’t base any official configuration handling off this particular library. The terraform_config_inspect library was also originally created to deal with all versions of Terraform even back to < v0.12, which contributes to many of the additional quirks since even the HCL language was quite different at that time.

Short of some invalid handling of data, both sides of this are unfortunately what they are. The terraform_config_inspect library was made long ago to fit a particular role of extracting a high level overview of a config and fits that role for current consumers. The Terraform CLI is hampered by a compatibility guarantee that we won’t change the behavior for configuration that is working. If there is something causing a problem in Terraform CLI usage, we can try to address it, maybe with a warning or better documentation.

In the long term, I have notes of all these various inconsistent parts within Terraform which we can try and address when we can finally make breaking changes, but for now there really isn’t anything to reconcile.