Is anyone aware of how to instantiate dynamic providers

Terraform
1

I have a case where I need to create multiple namespaces in the vault and I have to write down separate providers for each namespace. Is there any workaround available for this?

2

Provider configurations in Terraform must be statically-defined because they are one object in Terraform which is needed for essentially all operations, including planning and destroying resources. In particular, a provider configuration must always outlive all of the resource instances that belong to it, and so must survive for at least one additional terraform apply than the associated resource configurations do.

However, you can use a shared module called multiple times as a way to factor out everything except the provider configuration so it can be maintained in a central location.

For example, you could create a module in ./modules/vault-namespace (just for example) and call it once for each namespace, each one coupled with its own provider configuration, like this:

provider "vault" {
  alias = "a"

  # ...
}

module "a" {
  source = "./modules/vault-namespace"

  # (any required settings)

  providers = {
    vault = vault.a
  }
}

provider "vault" {
  alias = "b"

  # ...
}

module "b" {
  source = "./modules/vault-namespace"

  # (any required settings)

  providers = {
    vault = vault.b
  }
}

The providers argument inside a module block tells Terraform to give the called module a different “view” of the available provider configurations than the caller had. In this case, the first module call says that the default vault provider configuration in the child module maps to the “a”-aliased configuration from the root module, whereas the second one uses the “b”-aliased configuration.

In order to meet the requirement I mentioned above about resource instances always outliving their provider configurations, if you ever want to remove one of your namespaces that would require a two step process. For example, to remove the “b” namespace:

  • Remove the module "b" block, and apply that change to destroy all of the objects it declared.
  • Now remove the provider "vault" block that has alias = "b", which no longer has any resource instances associated with it. You can run terraform plan to confirm that there are no remaining references to that provider configuration.

The addition of a new namespace can typically happen in only a single step though, because when you first add the new module block and associated provider "vault" block the provider configuration will be available to support the initial creation of the objects declared in that module.

2 Likes
3

Not sure if I should create my own thread for this but my question is very similar to OPs.

I currently have a lot of terraform code that creates a bunch of GKE clusters for various dev teams. I’d like to bootstrap these clusters by using terraform to deploy Kubernetes manifests. But since I use modules and for_each I can’t use a kubernetes provider within the module. But I also can’t pass in the Kubernetes provider details into the module, such the the host IP, as the cluster doesn’t exist when I call the module.

My final attempt at an approach was to have the cluster details as an output from the module that creates the cluster and pass it into a new cluster_bootstrap module that deploys the manifest. But this can’t work, as the provider config that’s passed in has to be static.

I’d love to be able to do something like this:

module "fluxcd" {
  for_each = module.cluster

  providers = {
    kubernetes = {
      host  = each.endpoint
      client_certificate     = each.client_certificate
      client_key             = each.client_key
      cluster_ca_certificate = each.cluster_ca_certificate
    }
  }
  
}

Is there anything on the roadmap to support something like this?

2 Likes
4

Ok, I see this is being tracked here:

And here:

1 Like
5

I’m looking at a use case where your ‘module’ snippet perfectly describes what I’d love to do, but it is not supported by Terraform