Terraform reusable modules and provider declarations best practices

Hello, have a question around reusable tf modules and declaration of providers. Can you please recommend some best practices on how to declare/pass providers during consumption Of these modules via CI/CD pipelines.

We have few providers declared inside these reusable modules which is not working for us. We are not able to use depends_on or some other features as modules has embedded provider declaration

For example:

The default provider we are passing from the parent/root module so that resources without any provider declaration will inherit them, however any additional required providers we are declaring inside these modules.

`Module1 : 
      Inside the root level main.tf has the following provider declaration
  provider "aws" {
      alias  = "secondary"
     region = var.secondary_aws_region
  }

Module2:
   Inside the root level main.tf has the following provider declaration
   Provider “postgresql”

Module3:
 Provider Vmware
`

Questions

  1. What is the best way to pass or declare the providers when developing the reusable modules?
  2. I was trying to follow the provider documentation (Provider Configuration - Configuration Language | Terraform by HashiCorp) , didn’t quite get the when to use the configuration_aliases
  3. If you can provide few simple examples that will be helpful

Regards
Raj

Hello @rxxk-cg,

As a general rule, only root modules (the directory where you run terraform apply) should have provider blocks. Shared modules should not have provider blocks, and should instead either inherit or have explicitly passed provider configurations from the root.

For a shared module which only needs one configuration for a particular provider it’s sufficient to just declare a dependency on the provider itself, and therefore just inherit the default (unaliased) configuration for that provider from the root module, without any explicit extra wiring on the module user’s part:

terraform {
  required_providers {
    postgresql = {
      # I'm assuming that cyrilgdn's provider is the
      # one you meant when you said "postgresql";
      # specify a different username here if not.
      source = "cyrilgdn/postgresql"
    }
  }
}

As long as the root module includes a configuration for cyrilgdn/postgresql, the child module will automatically inherit it from the root without any explicit passing by the caller. I would recomment using this approach in most cases; the rest of this post is about a less common situation.


The extra configuration_aliases argument deals with the more complex situation where the provider needs to use more than one configuration for the same provider, in which case configuration_aliases declares which alias names the module will use:

terraform {
  required_providers {
    postgresql = {
      source = "cyrilgdn/postgresql"

      configuration_aliases = [
        postgres.a,
        postgres.b,
      ]
    }
  }
}

The above declares two alternate (“aliased”) provider configurations called postgres.a and postgres.b. You can then select one of those configurations for each of the resources in the module which is implemented by that provider.

In this situation with multiple provider configurations, Terraform can’t automatically infer the meaning of the aliases a and b and so instead of automatic inheritance of the default provider configurations the users of your module will need to specify which of their own provider configurations correlates with each one. For example, if the caller of your module is a root module which declares its own concrete provider configurations foo and bar then it could associate them with your module’s a and b like this:

terraform {
  required_providers {
    postgresql = {
      # NOTE: The caller and callee must both agree
      # about which provider source they are using,
      # since this is how Terraform knows that both
      # modules are using the same provider.
      source = "cyrilgdn/postgresql"
    }
  }
}

provider "postgres" {
  alias = "foo"

  # ...
}

provider "postgres" {
  alias = "bar"

  # ...
}

module "example" {
  source = "../modules/example"
  providers = {
    postgres.a = postgres.foo
    postgres.b = postgres.bar
  }

  # ...
}

The above module block’s providers argument specifies that the child module’s a configuration correlates with the root module’s foo, and the child module’s b correlates with bar.

As I mentioned above, I would suggest not using this more complicated approach unless your module is specifically representing some sort of connection between two configurations. For example, a module which sets up a peering connection between two AWS VPCs in different accounts or regions naturally ends up needing two provider configurations due to how the AWS provider is designed: account and region are provider-configuration-level settings rather than resource arguments. But I would typically separate that particular need into its own focused module, and then let the rest of the problem be described with simpler modules that only work with one configuration at a time.

Thank you @apparentlymart , I followed your suggestion and revised the code

Below the declaration of my root level main.tf


*********************rootlevel main.tf*********************

provider "aws" {
		region = var.aws_region    
		}

provider "aws" {
		alias = "secondary"
		region = var.secondary_aws_region   
		}

Below the module i am invoking to deploy some RDS hosted databases , in this case i have resources inside the module with explict secondary  
 
module "rds-database" {
  source  = "tfe.xxx.com/xxgregistry/rds-db/aws"
  version = "4.0.5"
  
  <** 
   required variables
  **>
  
  providers = {
        aws.secondary = aws.secondary
    }
}

*********************end of rootlevel main.tf*********************

Inside the module i have a versions file with the following declaration

 terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.71.0"     
    }

    random = {
      source  = "hashicorp/random"
      version = ">= 3.1.0"
    }
  }
}

during the terraform init at the root level i am running into the following issue


â•·
â"‚ Warning: Provider aws.secondary is undefined
â"‚ 
â"‚   on main.tf line 56, in module "rds-database":
â"‚   56:         aws.secondary = aws.secondary
â"‚ 
â"‚ Module module.rds-database does not declare a provider named aws.secondary.
â"‚ If you wish to specify a provider configuration for the module, add an entry for aws.secondary in the required_providers block within the module.
╵

Terraform has been successfully initialized!

so i changed the my inside the module versions.tf file where i declared the required_providers

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 3.71.0"
      configuration_aliases = [ 
          aws.secondary
       ]
    }

    random = {
      source  = "hashicorp/random"
      version = ">= 3.1.0"
    }
  }
}

Now its working fine with my root level maint.tf file. However here is the issue

A user can leverage this module to deploy resources in a single region(defaut aws provider) or multiple regions (secondary alias provider).

As i declared the secondary provider in the configuration_aliases , now we are forced to supply both default aws and secondary provider, even though user is cosuming this module to deploy resource only with default aws/singe region resources

How can i remove this constraint ?

Regards
Raj