Hi!
I’m currently creating an array of resources using the for_each construct. Here’s an example:
# variables.tf
variable "sub_environments" {
type = list(string)
description = "List of sub-environments we'd like to create"
default = ["one", "two"]
}
# resource.tf
resource "my_resource" "hi" {
for_each = toset(var.sub_environments)
name = "thing-${each.value}-resource"
... etc ...
}
This works great, and I successfully get two resources created. My problem, however, is that this code is in a re-usable module, and there are times when I do not want multiple resources to be made. I only want one resource made.
If I make the sub_environments variable above to be an empty list, Terraform thinks that nothing should be created. I also don’t want to pass in a single item to sub_environments (e.x. sub_environments = ["one"] because I don’t want my resource name to be thing-one-resource, I just want thing-resource).
Are there any tricks I can do to be able to pass in an empty list to sub_environments and have only one resource be created?
Thanks!
Hi @johnernaut,
Terraform doesn’t have any built-in features for treating a single item as a special case – usually one item is treated the same as any N of items – but you can do it by writing out conditional expressions that reflect the special cases you want to make.
You only indicated that the name should be different when there’s only one. You could do that by setting name with a conditional expression, like this:
variable "sub_environments" {
type = set(string)
description = "Set of sub-environments we'd like to create"
default = ["one", "two"]
}
locals {
# Set to true if we're in "single environment" mode
# which activates some special cases.
single_env = length(var.sub_environments) == 1
}
resource "my_resource" "hi" {
for_each = var.sub_environments
name = local.single_env ? "thing-resource" : "thing-${each.value}-resource"
# ...
}
You can use local.single_env as the condition for any other case you want to treat differently when there’s only one environment.
@apparentlymart Thanks so much for the response! As you mentioned above, I only indicated the name field above, but that was in the spirit of brevity.
In reality my resource references many other resources that also use a for_each on var.sub_environments, so a more accurate representation of the resource looks like this:
resource "my_resource" "hi" {
for_each = toset(var.sub_environments)
name = "thing-${each.value}-resource"
# this property relies on another resource that uses a `for_each` as well
task_definition = "${aws_ecs_task_definition.rails-task-definition[each.value].family}"
# this block has to be dynamic because I'm using a for_each
# without a for_each (under some condition), this block would need to look different -
# like load_balancer { } - because I'd only have 1
dynamic "load_balancer" {
for_each = toset(var.sub_environments)
content {
container_name = "nginx"
container_port = "80"
target_group_arn = aws_lb_target_group.ecs_rails_target_group[each.value].arn
}
}
}
So ultimately I’d like to be able to create a single resource (and single associated resources) when the sub_environments list is empty. I’m wondering if I just need to somehow create completely separate resource blocks to use when that’s the case, but I’m not sure about the best way to accomplish that…
Completely separate resource blocks would work, but I expect it would not be very maintainable.
My instinct here would be to continue down the path I started in my previous answer. I hadn’t quite understood that you wanted to treat "no sub_environments" as the special case, rather than just one. The single item and multiple item cases still look more similar than different to me: there are a few cosmetic differences in naming, but the overall structure is the same in both cases.
variable "sub_environments" {
type = set(string)
description = "Set of sub-environments we'd like to create"
default = []
}
locals {
# each_env is our for_each value for anything that
# ought to be replicated per environment when
# sub_environments are set. It's a single-element
# set containing just empty string (a placeholder)
# when no sub-environments are enabled, because
# we only want to create one instance per resource
# in that case.
single_env = length(var.sub_environments) == 0
each_env = local.single_env ? toset([""]) : var.sub_environments
}
resource "my_resource" "hi" {
for_each = local.each_env
name = local.single_env ? "thing-resource" : "thing-${each.value}-resource"
# this property relies on another resource that uses a `for_each` as well
task_definition = aws_ecs_task_definition.rails-task-definition[each.key].family
dynamic "load_balancer" {
for_each = local.each_env
content {
container_name = "nginx"
container_port = "80"
target_group_arn = aws_lb_target_group.ecs_rails_target_group[each.key].arn
}
}
}
As long as you use local.each_env consistently for all of the resource for_each expressions that need this special behavior of switching between 1 or N then the references to them should all line up, because in the single-instance case each.key will consistently be "" and so the lookup should succeed.
The one little quirk with this is that in the single case the instances will have addresses like my_resource.hi[""], which is a little unusual but I think a reasonable tradeoff to keep the module implementation relatively straightforward. (If the empty string seems uncomfortable then you could set the default value to any other string.
Thanks for the information @apparentlymart! I’m starting to implement your suggestion (which I really like), but there are some errors I’m running into. Particularly this one which is raised on this line in my resource:
task_definition = aws_ecs_task_definition.rails-task-definition[each.key].family
Error:
Error: Invalid index
on ../../../modules/services/rails/ecs.tf line 73, in resource "aws_ecs_service" "sidekiq-svc":
73: task_definition = aws_ecs_task_definition.rails-task-definition[each.key].family
|----------------
| data.aws_ecs_task_definition.sidekiq is object with no attributes
| each.key is ""
It seems like I can’t use a blank string as an item. Any ideas?
Thanks!
I think the subtle clue in that error message is this statement:
data.aws_ecs_task_definition.sidekiq is object with no attributes
A resource with for_each set would only appear as “object with no attributes” if its for_each expression were an empty collection. That makes me suspect that the for_each inside the data "aws_ecs_task_definition" "sidekiq" block is still var.sub_environments rather than local.each_env. Is that true?
@apparentlymart right you are! I did go ahead and update the data block so that it uses local_env:
data "aws_ecs_task_definition" "sidekiq" {
for_each = local.each_env
task_definition = aws_ecs_task_definition.sidekiq-task-definition[each.key].family
}
Unfortunately though, it gives this error which makes it seem like it can’t find the correct resource:
Error: Failed getting task definition ClientException: Unable to describe task definition. "g4-staging-sidekiq-ecs-task-family"
on ../../../modules/services/rails/data.tf line 138, in data "aws_ecs_task_definition" "sidekiq":
138: data "aws_ecs_task_definition" "sidekiq" {
However, it seems like it should be able to match "g4-staging-sidekiq-ecs-task-family" since I do have that name set in the resource:
resource "aws_ecs_task_definition" "sidekiq-task-definition" {
family = local.single_env ? "g4-staging-sidekiq-ecs-task-family" : "g4-staging-${each.key}-sidekiq-ecs-task-family"
}
Hi @johnernaut,
I think this new error is an order of operations problem rather than a for_each / conditional problem. It’s unusual to both manage and read the same object in a single configuration, and if you do that you need to take some extra care because by default a data resource will be read during the planning phase, before the object has been created.
The best answer here, if possible, is to remove the data "aws_ecs_task_definition" "sidekiq" block altogether and make anything that refers to data.aws_ecs_task_definition.sidekiq refer directly to aws_ecs_task_definition.sidekiq-task-definition instead. That will then cause Terraform to properly understand the dependency relationships, ensuring that resources which rely on the task definition will only be processed after the task definition is created.
In an unusual case where you do need to both read and manage the same object in the same configuration, you need to find some way to ensure that Terraform can understand what order of operations is needed. In Terraform 0.12 the only way to do that is to include in the data resource configuration some value from the corresponding resource that is unknown during planning, which is annoying in this particular case because it looks like family is a value set in your configuration and thus it will always be known during planning.
The forthcoming Terraform v0.13.0 release (current in beta) introduces a slightly easier answer: you can use depends_on to tell Terraform that any action related to that data resource must wait until any actions related to the explicit dependencies are complete:
data "aws_ecs_task_definition" "sidekiq" {
for_each = local.each_env
depends_on = [aws_ecs_task_definition.sidekiq-task-definition]
task_definition = aws_ecs_task_definition.sidekiq-task-definition[each.key].family
}
depends_on in a data block takes on a slightly special behavior in Terraform 0.13: if (and only if) there is any change in the plan for aws_ecs_task_definition.sidekiq-task-definition then Terraform will wait until the apply phase to read the data source. If there are no pending changes to aws_ecs_task_definition.sidekiq-task-definition then Terraform will read the data source during planning as normal.
With all of that said, I’d still recommend eliminating the data resource altogether if possible, because the result will be simpler and thus easier to follow for future maintainers.
3 Likes
@apparentlymart Got it. I ended up applying depends_on in the data block like you suggested, and everything appears to be working properly.
Thanks so much for you help!