resource "google_project_service" "pubsub-service" {
project = "${var.project-id}"
service = "pubsub.googleapis.com"
# Waiting until pubsub api is enabled and linked with organization policy. Waiting 60sec is minimum.
provisioner "local-exec" {
command = "sleep 60"
}
}
resource "google_pubsub_topic" "topic" {
name = "${var.topic["name"]}"
project = "${var.project-id}"
depends_on = ["google_project_service.pubsub-service"]
}
resource "google_pubsub_topic_iam_member" "publisher" {
count = "${length(var.topic["publisher"])}"
topic = "projects/${var.project-id}/topics/${var.topic["name"]}"
role = "roles/pubsub.publisher"
member = "serviceAccount:${element(var.topic["publisher"], count.index)}@${var.project-id}.iam.gserviceaccount.com"
depends_on = ["google_pubsub_topic.topic"]
}
resource "google_pubsub_subscription" "subscription" {
count = "${length(keys(var.topic["subscriptions"]))}"
name = "${element(keys(var.topic["subscriptions"]), count.index)}"
topic = "projects/${var.project-id}/topics/${var.topic["name"]}"
project = "${var.project-id}"
depends_on = ["google_pubsub_topic.topic"]
}
##### Everything until here works #####
# This is just the template form
# Here I want to iterate through the subscriber list depending on the subscription and add serviceAccounts to the corresponding subscription
resource "google_pubsub_subscription_iam_member" "subscriber" {
subscription = "your-subscription-name"
role = "roles/pubsub.subscriber"
member = "serviceAccount:jane@example.com"
depends_on = ["google_pubsub_subscription.subscription"]
}
The problem: I don’t how to iterate through the nested list in the subscriptions map in the variables.tfwith the subscriber resource in the main.tf. The form in the variables.tf is the easiest form I can think of for my colleagues, customers and myself because I have everything in one structure and no duplicates.
The only alternative would be to separate topics and subscriptions, but then I have to maintain both forms and have duplicates because I have to specify the topics at the subscriptions form.
I would be very glad if someone could help me with this. Maybe anyone could make an example how I can proceed.
Maybe you can make use of the for_each syntax available in 0.12 (https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each) to loop over the var.my-topic-1.subscriptions and then each.key being the “subscription-1”, “subscription-2”, etc. and each.value being the list of subscribers. I think that might work, but don’t have a computer to test it on near me at the moment.
Something like this:
resource "google_pubsub_subscription_iam_member" "subscriber" {
for_each = var.my-topic-1.subscriptions
subscription = each.key
member = each.value //this would be the list. I dont know if you are able to add multiple members at once.
}
I also found this link earlier and I tried to implement it but no success (sorry that I didn’t mention that). I don’t know how to implement it correctly I guess. There is no documentation available yet.
My main.tf looks like this:
resource "google_project_service" "pubsub-service" {
project = var.project-id
service = "pubsub.googleapis.com"
provisioner "local-exec" {
command = "sleep 60"
}
}
resource "google_pubsub_topic" "topic" {
name = var.topic["name"]
project = var.project-id
depends_on = ["google_project_service.pubsub-service"]
}
resource "google_pubsub_topic_iam_member" "publisher" {
count = length(var.topic["publisher"])
topic = "projects/${var.project-id}/topics/${var.topic["name"]}"
role = "roles/pubsub.publisher"
member = "serviceAccount:${element(var.topic["publisher"], count.index)}@${var.project-id}.iam.gserviceaccount.com"
depends_on = ["google_pubsub_topic.topic"]
}
resource "google_pubsub_subscription" "subscription" {
count = length(keys(var.topic["subscriptions"]))
name = element(keys(var.topic["subscriptions"]), count.index)
topic = "projects/${var.project-id}/topics/${var.topic["name"]}"
project = var.project-id
depends_on = ["google_pubsub_topic.topic"]
}
resource "google_pubsub_subscription_iam_member" "subscriber" {
for_each = var.my-topic-1.subscriptions # doesn't work with var.topic.subscriptions either
subscription = each.key
role = "roles/pubsub.subscriber"
member = "serviceAccount:${each.value}@${var.project-id}.iam.gserviceaccount.com" # does not work with ${each.value.subscription-1} either
depends_on = ["google_pubsub_subscription.subscription"]
}
After executing terrafom plan or terraform apply I get the following output:
Error: Reference to undeclared resource
on modules/resources/pubsub/main.tf line 35, in resource "google_pubsub_subscription_iam_member" "subscriber":
35: subscription = each.key
A managed resource "each" "key" has not been declared in my-project-1.topic-1.
Error: Reference to undeclared resource
on modules/resources/pubsub/main.tf line 37, in resource "google_pubsub_subscription_iam_member" "subscriber":
37: member = "serviceAccount:${each.value.subscription-1}@${var.project-id}.iam.gserviceaccount.com"
A managed resource "each" "value" has not been declared in
my-project-1.topic-1.
I changed the google_pubsub_subscription_iam_member resource to google_pubsub_subscription_iam_binding in order to pass a list of subscribers to the members attribute. This is the only way I could think of to create a set of members to an IAM role while preserving the initial data model .
I use the google_pubsub_subscription.subscription resources as the iterator for the IAM binding. This is to represent your idea that for each subscription, we have a list of subscribers. Rather than iterate over the original variable, it is sometimes easier to iterate over the created resources and reference the variable map.
This yields the following (excerpted) output when running terraform plan:
# google_pubsub_subscription_iam_binding.subscriber[0] will be created
+ resource "google_pubsub_subscription_iam_binding" "subscriber" {
+ etag = (known after apply)
+ id = (known after apply)
+ members = [
+ "serviceAccount:subscriber-1@my-project-id.iam.gserviceaccount.com",
+ "serviceAccount:subscriber-2@my-project-id.iam.gserviceaccount.com",
]
+ project = (known after apply)
+ role = "roles/pubsub.subscriber"
+ subscription = "subscription-1"
}
# google_pubsub_subscription_iam_binding.subscriber[1] will be created
+ resource "google_pubsub_subscription_iam_binding" "subscriber" {
+ etag = (known after apply)
+ id = (known after apply)
+ members = [
+ "serviceAccount:subscriber-2@my-project-id.iam.gserviceaccount.com",
+ "serviceAccount:subscriber-3@my-project-id.iam.gserviceaccount.com",
]
+ project = (known after apply)
+ role = "roles/pubsub.subscriber"
+ subscription = "subscription-2"
}
Thank you very much for your help. This helped a lot! Now it works like expected and this topic/problem is solved for me.
Please answer me the following two questions related to your reply for understanding purpose:
I recognized that the _, after your for has no effect. Is there a meaning to it?
And to your second note point: Do I understand you right, when I define one or multiple topics, that terraform creates an iterable list of these resources that you use to specify the subscription?
I recognized that the _, after your for has no effect. Is there a meaning to it?
This would usually denote the index of the element in the list, if you want to use it. I think you can omit it (give it a try).
And to your second note point: Do I understand you right, when I define one or multiple topics, that terraform creates an iterable list of these resources that you use to specify the subscription?
If you use count to create resources iteratively, it will generate a list of resources created and their attributes. If you do not use count, you will only be able to access the specific resource with its identifier.