Refer Modules with Different Terraform Version State file

Team,

I have a situation where my current module say SQS (Terraform version : 0.12.29) wants to refer another KMS module (terraform version: 0.12.26) to fetch KMS Key alias via data source. Since both are on different version of terraform, how I solve this problem here.

SQS Source

module "kms" {
  source = "./modules/resolver/kms"
  

  tfstate_bucket_name = var.tfstate_bucket_name
  tfstate_kms_key = var.tfstate_kms_key
  region = var.region
  profile = var.profile
}
resource "aws_sqs_queue" "this" {
  .
  .
  .
  .
  kms_master_key_id                 = module.kms.kms-key-alias  # This is KMS key alias created in another state file.
}

KMS Source

data "terraform_remote_state" "kms" {
  backend = "s3"
  config = {
    bucket  = var.tfstate_bucket_name
    key     = var.tfstate_kms_key
    region  = var.region
    profile = var.profile
  }
}

The error I eventually land up

Error: Unsupported Terraform Core version

  on versions.tf line 3, in terraform:
   3:   required_version = "0.12.26"

In fact this is become a common problem for me, where there are several modules written in various version of terraform and I want to fetch data from those in my current module. This is making difficult to allow me to use latest version of terraform.

Expected Solution:
If there is way to change the terraform version before the calling that specific module and revert to the terraform version when creating resources in the current module.

Any suggestions are most welcome

@apparentlymart any feedback or suggestions are really appriciated.

+1, I am also facing the same issue where dependent TF modules are not able to upgrade as underlying modules are on diff TF versions.

1 Like

Hi @smitjainsj,

I would typically recommend against using exact version constraints like that in a shared module. An exact version constraint can be useful in a root module in order to specify exactly which Terraform version your team is currently using with it, but for a shared module it’s troublesome for the situation you saw here: now everything needs to change together whenever you upgrade.

Instead, shared modules typically specify just a minimum Terraform version containing the features they depend on:

terraform {
  required_version = ">= 0.12.0"
}

This then allows a potential user of the module to see that it won’t work with, in this case, Terraform 0.11 or earlier, but it doesn’t prevent that user from upgrading to a newer Terraform version.

Since it seems like both of these modules are under your control (they both live in the same repository/package), hopefully you can change the shared module to specify only a minimum version.

An exact version constraint for the root module can often be too limiting too, since if you have lots of configurations you’d need to update each of them each time you upgrade Terraform. Instead, you might prefer to compromise and constrain only to a particular Terraform release while allowing patch version upgrades:

terraform {
  required_version = "~> 0.12.0"
}

This constraint will allow any 0.12.x release but will reject 0.13.x and later releases.

thanks @apparentlymart for the advice, Ill change the version.tf to allow minor version bump increase to maintain compatibility within the same version.

However, for shared modules how do you suggest ?

For eg: Shared-module-A is on 0.12.26 and Module-B is 0.12.29 . Is there a way to switch the terraform version to like 0.12.x when referring to Shared-module-A state file or B from a module C

modules
β”œβ”€β”€ main.tf
β”œβ”€β”€ resolver
β”‚   β”œβ”€β”€ A(0.12.26)
β”‚   β”‚   β”œβ”€β”€ main.tf
β”‚   β”‚   β”œβ”€β”€ provider.tf
β”‚   β”‚   β”œβ”€β”€ output.tf
β”‚   β”‚   └── version.tf
β”‚   └── B (0.12.29)
β”‚       β”œβ”€β”€ main.tf
β”‚       β”œβ”€β”€ provider.tf
β”‚       β”œβ”€β”€ output.tf
β”‚       └── version.tf
└── C (0.13.x)
    β”œβ”€β”€ main.tf
    β”œβ”€β”€ output.tf
    └── variables.tf

root(main.tf)

module "A" {
  source = "./resolver/A"
 ....
}

module "B" {
  source = "./resolver/B"
 ...
}

# This is where is fails for me in Module C as Module A and Module B both are shared modules and on different versions of Terraform.
module "C" { 
 source = "./C"
 output_from_A = <blah blah>
 output_from_B = <blah blah>
....
}

If they are kept as exact matches you won’t be able to run Terraform at all. You should adjust all the version requirements to be >= or ~> as @apparentlymart recommends.

@stuart-c that will only comply when I a higher minor version of TF module calling a same or lower minor version module and not the other way round. Also, it does not work with lower to higher versions in minor and major versions both.

Therefore, there should be a feature to switch TF before making calls to a TF Module which on a different version and it would be easy to control via some file structure or variable defined.

Generally you’d set your modules to only require a minimum version of a provider/Terraform, based on the functionality you need (e.g. needs AWS provider of at least X and needs 0.12 or later for Terraform).

It is then simple to fulfil those version constraints.

Terraform is a single command which checks all versions constraints & loads providers at startup, so there is no way to handle incompatible requirements.

I’m not seeing any difficulties in updating your modules to all have >= constraints rather than requiring a specific version?