Specifying different providers in a resource using "for_each"

Terraform
1

I’m trying to create a postgres user management configuration, and I’ve got several independent postgres instances across which to manage users.

I’d like to be able to have a single module for the “user”, and loop through each provider (postgres instance), creating the user with the given username and password and specific roles per instance without adding a new module for each instance-user combination, but I can’t figure out how to specify a different provider for each instance of a resource/module using “for_each”.

I’ve even upgraded to 0.13-beta-2, to try using “for_each” on the module, since I had seen examples defining module providers as a string, but apparently those were typos, because I just get an error saying providers can’t be specified in quotes.

Here’s what I currently have that doesn’t work:

provider "postgresql" {
  alias "instance_a"
  ...
}
provider "postgresql" {
  alias "instance_b"
  ...
}
provider "postgresql" {
  alias "instance_c"
  ...
}

module "user_joe" {
  source = "../../modules/postgres-users"
  for_each = {
    instance_a = [a_roles.read_only["dbname"]]
    instance_b = [b_roles.admin["dbname2"]]
  }
  providers = {
    postgresql = "postgresql.${each.key}"
  }

  username = "joe"
  password = var.joes_password
  roles = each.value
}

It seems like this would be a useful pattern to allow for, given that certain providers refer to infrastructure that might be array-like in nature. Any ideas on how to approach this?

2

Hi @mltsy,

Unfortunately this is a design pattern that Terraform cannot support at this time. The only way to achieve something like this is via code generation techniques, to generate all of the necessary provider and module blocks as a preprocessing step.

It’s likely that this situation will improve in future versions of Terraform, but addressing this problem was not part of the scope for Terraform 0.13 and there hasn’t been any detailed design work for dynamic provider configuration association (which is what this use-case requires) so I can’t say at this point when such a thing might become possible.

Rather than waiting for this capability for which there is no specific plan yet, I’d suggest either using code generation in a wrapper program around Terraform that is specialized to your goals, or using some other software to solve this problem.

3

Thanks for the response! Very helpful - I mean, not the answer I was hoping for, but I did expect as much :wink: I think for now, I’ll just live with redundant resources, and see how bad it gets.

Code generation is an interesting thought though - do you have any recommendations for good TF code generators that might be helpful here?

Let me also just say, I really appreciate your helpful and thorough responses! Thanks a lot for helping to make terraform 0.x a friendly tool :slight_smile:

4

I don’t know of any general solutions but I can say that in a previous life (when I was a Terraform user rather than a Terraform developer) I wrote a little Python program that read in some data from a simpler configuration file and wrote out Terraform JSON syntax because that was relatively easy to generate using Python’s json library.

My use-case then was somewhat similar to yours: the input data was a list of users and a list of groups, and the generated code was intended to project those users and groups (along with the user group memberships) into a variety of different target systems such as AWS IAM, the CI system we were using, etc.

I was doing it to compensate for Terraform not having for_each yet at all rather than specifically for dynamic provider configurations, but the same principle should apply to what you are doing. You’d probably generate JSON looking something like this:

{
  "provider": {
    "postgresql": [
      {"alias": "instance_a"},
      {"alias": "instance_b"},
      {"alias": "instance_c"},
    ]
  },
  "module": {
    "user_joe_instance_a": {
      "source": "../../modules/postgres-users",
      "providers": {"postgresql": "postgresql.instance_a"},
      "username": "joe",
      "roles": "${var.a_roles.read_only[\"dbname\"]}"
    },
    "user_joe_instance_b": {
      "source": "../../modules/postgres-users",
      "providers": {"postgresql": "postgresql.instance_b"},
      "username": "joe",
      "roles": "${var.b_roles.admin[\"dbname2\"]}"
    }
  }
}