Is anyone aware of how to instantiate dynamic providers

irab · June 19, 2022, 5:04am

Ok, I see this is being tracked here:

Dynamic provider configuration assignment

opened 03:27AM - 15 Jun 20 UTC

enhancement config

### Current Terraform Version  ``` Terraform v0.13.0-beta1 ``` ### Use-cases  It would be nice to be able to create dynamic providers. The main reason for my usage would be for aws assume_role. I have Terraform to create a number of AWS Subaccounts, and then I want to configure those subaccounts in one apply, instead of breaking them up across multiple apply steps. Currently this is done via modules, but with 0.12, I had to manually define copies of the modules for each subaccount. As said by the 0.13 modules doc: https://github.com/hashicorp/terraform/blob/master/website/docs/configuration/modules.html.md#limitations-when-using-module-expansion > Modules using count or for_each cannot include configured provider blocks within the module. Only proxy configuration blocks are allowed. ### Attempted Solutions  ```terraform # Organisational Group for Crimson App. module "org_crimson_app" { source = "./modules/organisation-group" organisation_id = local.root_organisation_id group_name = "crimson-app" billing_alert_emails = ["<billing account>"] accounts = { production = {} staging = {} test = {} } zones = { public = { create = true zone_id = "" domain = "app.crimsoneducation.org" } internal = { create = true zone_id = "" domain = "app.crimsoneducation.io" } } master_role_name = local.organisation_root_role account_owner_username = var.account_owner_username account_owner_domain = var.account_owner_domain // Workaround for https://github.com/hashicorp/terraform/issues/17519 account_configuration = { production = module.org_crimson_app_production_config.config staging = module.org_crimson_app_staging_config.config test = module.org_crimson_app_test_config.config } } // Workaround for https://github.com/hashicorp/terraform/issues/17519 module "org_crimson_app_production_config" { source = "./modules/organisation-account-config" account_info = module.org_crimson_app.account_information.production } // Workaround for https://github.com/hashicorp/terraform/issues/17519 module "org_crimson_app_staging_config" { source = "./modules/organisation-account-config" account_info = module.org_crimson_app.account_information.staging } // Workaround for https://github.com/hashicorp/terraform/issues/17519 module "org_crimson_app_test_config" { source = "./modules/organisation-account-config" account_info = module.org_crimson_app.account_information.test } ``` Source for `./modules/organisation-group`: ```terraform # The Organisational Unit for this group. resource "aws_organizations_organizational_unit" "unit" { name = var.group_name parent_id = var.organisation_id lifecycle { prevent_destroy = true } } # The environment accounts. resource "aws_organizations_account" "environments" { for_each = var.accounts parent_id = aws_organizations_organizational_unit.unit.id # Use account_name if provided, otherwise [group]-[account]. name = local.account_full_name[each.key] email = "${var.account_owner_username}+aws-org-${local.account_full_name[each.key]}@${var.account_owner_domain}" role_name = var.master_role_name lifecycle { # There is no AWS Organizations API for reading role_name ignore_changes = [role_name] prevent_destroy = true } } # Collect all Account Names locals { account_full_name = { for account_name, account in var.accounts : account_name => lookup(account, "account_name", "${var.group_name}-${account_name}") } } ... ``` Source for `./modules/organisation-account-config`: ```terraform // Workaround for https://github.com/hashicorp/terraform/issues/17519 variable "account_info" { type = object({ group_name = string account_name = string account_full_name = string alias = string master_role_arn = string zones = map(object({ zone_name = string fqdn = string })) add_terraform_bucket = bool billing_alert_amount = number billing_alert_currency = string billing_alert_emails = list(string) }) } # Create Provisioner Assuming Child Account Role. provider "aws" { region = "ap-southeast-2" assume_role { role_arn = var.account_info.master_role_arn } } resource "aws_iam_account_alias" "alias" { account_alias = var.account_info.alias } # Bucket for Terraform State. resource "aws_s3_bucket" "terraform_state" { count = var.account_info.add_terraform_bucket ? 1 : 0 bucket = "${var.account_info.account_full_name}-terraform" versioning { enabled = true } } # Fix for Issue: https://medium.com/faun/aws-eks-the-role-is-not-authorized-to-perform-ec2-describeaccountattributes-error-1c6474781b84 resource "aws_iam_service_linked_role" "elasticloadbalancing" { aws_service_name = "elasticloadbalancing.amazonaws.com" } ... ``` ### Proposal  Module `for_each`: ```terraform # Organisational Group for Crimson App. module "org_crimson_app" { source = "./modules/organisation-group" organisation_id = local.root_organisation_id group_name = "crimson-app" billing_alert_emails = ["<billing account>"] accounts = { production = {} staging = {} test = {} } zones = { public = { create = true zone_id = "" domain = "app.crimsoneducation.org" } internal = { create = true zone_id = "" domain = "app.crimsoneducation.io" } } master_role_name = local.organisation_root_role account_owner_username = var.account_owner_username account_owner_domain = var.account_owner_domain account_configuration = module.org_crimson_app_config } module "org_crimson_app_config" { for_each = module.org_crimson_app.account_information source = "./modules/organisation-account-config" account_info = each.value } ``` Provider `for_each`: This doesn't look as clean, but appeases the docs saying: > In all cases it is recommended to keep explicit provider configurations only in the root module and pass them (whether implicitly or explicitly) down to descendent modules. This avoids the provider configurations from being "lost" when descendent modules are removed from the configuration. It also allows the user of a configuration to determine which providers require credentials by inspecting only the root module. ```terraform # Organisational Group for Crimson App. module "org_crimson_app" { source = "./modules/organisation-group" organisation_id = local.root_organisation_id group_name = "crimson-app" billing_alert_emails = ["<billing account>"] accounts = { production = {} staging = {} test = {} } zones = { public = { create = true zone_id = "" domain = "app.crimsoneducation.org" } internal = { create = true zone_id = "" domain = "app.crimsoneducation.io" } } master_role_name = local.organisation_root_role account_owner_username = var.account_owner_username account_owner_domain = var.account_owner_domain account_configuration = module.org_crimson_app_config } provider "aws" { for_each = module.org_crimson_app.account_information alias = each.key assume_role { role_arn = each.value.master_role_arn } } module "org_crimson_app_config" { for_each = module.org_crimson_app.account_information source = "./modules/organisation-account-config" providers = { aws = aws[each.key] } account_info = each.value } ``` ### References  - Another user references a use case: https://discuss.hashicorp.com/t/terraform-v0-13-0-beta-program/9066/9 - https://github.com/hashicorp/terraform/issues/25320

And here:

github.com/hashicorp/terraform

Ability to pass providers to modules in for_each

opened 10:14PM - 26 Mar 20 UTC

mightyguava

enhancement providers

### Use-cases  I'd like to be able to provision the same set of resources in multiple regions a `for_each` on a module. However, looping over providers (which are tied to regions) is currently not supported. We deploy most of our infra in 2 regions in an active-passive configuration. So being able to instantiate both regions using the same module block would be a huuuge win. It's also our primary use case for for_each on modules being implemented in https://github.com/hashicorp/terraform/issues/17519. ### Proposal  Proposed syntax from @jakebiesinger-onduo ```tf provider "google" { alias = "goog-us-east1" region = "us-east1" } provider "google" { alias = "goog-us-west1" region = "us-west1" } locals { regions = toset(['us-east1', 'us-west1']) providers = { us-east1 = google.goog-us-east1 us-west1 = google.goog-us-west1 } } module "vpc" { for_each = local.regions providers = { google = local.providers[each.key] } ... } ``` Another option would be to de-couple the region from providers, and allow the region to be passed in to individual resources that are region aware. As far as I know, both AWS and GCP credentials at least are global. ### References  - count and for_each for modules https://github.com/hashicorp/terraform/issues/17519 - looping over providers https://github.com/terraform-providers/terraform-provider-bigip/issues/247 - Multiple regions with same Provider https://github.com/hashicorp/terraform/issues/451 - Reddit: https://www.reddit.com/r/Terraform/comments/cwp0d4/terraform_multi_region_question_can_i_just_use/

Topic		Replies	Views
Multiple Vault Namespaces in One Terraform Workspace / Configuration Set Vault hcp-terraform	4	1311	September 17, 2020
Terraform providers questions Terraform	1	150	June 3, 2024
Explicit Providers in Modules Terraform	3	710	September 23, 2020
Is it possible to configure the provider in a module dynamically? Terraform	5	1491	May 28, 2023
Dynamic providers inside modules Terraform	0	2410	March 18, 2020

Is anyone aware of how to instantiate dynamic providers

Related topics