Undocumented behavior around nested, aliased providers

Terraform
1

Hello! I ran into a problem with the way terraform 0.13 seems to handle Providers within Modules.

Specifically, the issue crops up when using a submodule with an implicitly-inherited, aliased provider. A reason this might come up is that the submodule needs two different google providers (to create resources in two google projects).

I have the following (example) root module and submodule:

# main.tf (root module, defines providers)
provider "google" {
  project = "my-primary-project"
}

provider "google" {
  alias = "other-project"
  project = "my-other-project"
}

module "some_submodule" {
  source = "./some_submodule"
}

# some_submodule/main.tf (submodule, uses both google providers)
provider "google" {
  alias = "my-other-project"
}

resource "google_service_account" "some_primary_project_resource" {
  provider = google
  
  # ... other config
}

resource "google_service_account" "some_other_project_resource" {
  provider = google.other-project
  
  # ... other config
}

Note that I am not explicitly passing down the other-project alias to the submodule. The docs say that this shouldn’t work:

Additional provider configurations (those with the alias argument set) are never inherited automatically by child modules, and so must always be passed explicitly using the providers map.

Given the documentation, I would expect this terraform config not to work at all. The submodule should fail with some kind of error like Provider not found: google.other-project. The submodule, which is not explicitly passed the aliased configuration, shouldn’t have access to that particular provider. Therefore some_submodule should not know to create some_other_project_resource in the my-other-project GCP project.

However, this config can actually be applied, and it leads to a messy state. Somehow the submodule is actually able to find the other-project alias (and therefore knows to create some_other_project_resource in the GCP project named my-other-project), but in terraform state, the resource’s provider is named module.some_submodule.providers[registry.terraform.io/hashicorp/google].other-project. So it is reusing the same provider configuration, but giving it a new name and tying it to the some_submodule module. This leads to some confusion, because now the some_submodule module cannot be deleted (because terraform thinks the provider definition is gone, even though it isn’t).

Is this actually a valid setup? My sense is that this configuration should cause an error to be thrown during terraform plan, perhaps with an error message that recommends adding a providers = { ... } argument to the some_submodule definition.

2

Hi @gkaemmer,

I think what happened here is that Terraform understood the provider "google" block in your child module as being a normal provider block rather than a proxy provider block, which happened to work because all of the arguments to that provider were either optional or satisfied by out-of-band settings like environment variables.

I agree that this behavior is confusing but it is an unfortunate consequence of how proxy provider syntax is ambiguous with a normal empty provider block, which is in turn a result of retrofitting the provider-passing behavior onto the existing Terraform language that didn’t have any first-class syntax for proxy provider configurations.

Given that an empty provider configuration is valid for a provider that doesn’t have any required arguments, I don’t think there’s any way for Terraform to return an explicit error here without making a change to the proxy provider syntax. We have considered changing that syntax in the past, but unfortunately our focus has been elsewhere and thus we’ve not been working on that part of the language since its last significant change in Terraform 0.11. :confounded:

3

Ah okay that actually makes a lot of sense, thanks for the response. I think you’re right about what was happening there.

One way to fix-without-fixing this would maybe be to more verbosely output the path of the provider that is being used to create each resource during a plan. I.e. something like this:

Terraform will perform the following actions:

  # module.some_submodule.google_service_account.some_other_project_resource will be created
  # NOTE: using non-root provider module.some_submodule.providers[registry.terraform.io/hashicorp/google].other-project
  + resource "google_service_account" "some_other_project_resource" {
      + email = (known after apply)
      + ...
    }

Might that be feasible?

4

Hi @gkaemmer,

That is an interesting idea indeed. Non-root providers are already deprecated in the documentation by explicitly recommending against them, and continue to be supported only for backward-compatibility, so what you proposed here could potentially be another step down the deprecation path, possibly leading to a later release where it’s an explicit Warning: and maybe even eventually being forbidden altogether (although there are some other language changes that would need to happen before that would be reasonable to do.)

We don’t really have any precedent for including provider addresses so prominently in the UI (except in a few error messages), so we might want to finesse that presentation a little to try to make it a bit more compact and readable, but I think the general idea has promise!