We have two modules. One deploys a security group and the other an OpenSearch domain. The OpenSearch uses the security group ID (an output) from the security group module.
Is there anyway to execute the below in a single apply without targeting the security group module first?
I hoped adding a depends_on to the opensearch call above would work, but it does not.
depends_on = [
module.sg
]
The error we receive:
│ Error: Invalid for_each argument
│
│ on .terraform/modules/opensearch/data.tf line 94, in data "aws_security_group" "es":
│ 94: for_each = var.es_security_group_ids
│ ├────────────────
│ │ var.es_security_group_ids is set of string with 1 element
│
│ The "for_each" set includes values derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that will identify the instances of this resource.
│
│ When working with unknown values in for_each, it's better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.
│
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully converge.
When considering your question, I think the key part of the error message you shared is the second paragraph:
When working with unknown values in for_each, it’s better to use a map value where the keys are defined statically in your configuration and where only the values contain apply-time results.
The error message suggests that module.sg.sg_id is a set of strings, and based on the name I’m going to guess that those strings are the id attributes from aws_security_group resource instances. Those values are not suitable to use as instance keys because they get decided by the remote system during the apply step, and so the AWS provider has no way to predict what the value will be during the planning step.
The error message is suggesting that you change this variable to be a map of strings instead, where the values are the security group IDs decided by the remote AWS API but the keys are values decided directly by your configuration.
In the module you’d declare the variable like this:
variable "es_security_group_ids" {
type = map(string)
}
In the module block in the calling module you’d assign it like this:
I’ve used “something” above because I don’t know from what you’ve shared what exactly this security group ID represents in your configuration. To decide what key might make more sense, I’d suggest thinking about the situation where there would be more than one element in this map and trying to describe what is different about them. If you know this particular caller will only ever pass one security group ID then the key doesn’t really matter and you can just set it to something constant.
If you name it something like I did above then Terraform will see that you intend to declare a resource instance with the following address:
data.aws_security_group.es["something"]
So as you can see as far as Terraform is concerned these keys don’t represent anything except just being a unique identifier to track each of the instances, so you can freely choose whatever key makes sense for the configuration you’re writing.
Thank you, and you are right in your assumptions. module.sg.sg_id is an output that will return only a single SG ID, but I could make multiple calls to module.sg to generate numerous SGs to pass to the OpenSearch module.
Using your recommendation I would then just need to pass them as the below, correct?
Yes, you can have as many elements in the map as you like as long as they all have unique keys. Each element of the map will correlate with a data.aws_security_group.es instance with the same key, so Terraform can use these keys to recognize which element of the input map correlates with each of your resources inside the module as you add and remove elements over time.