"The given key does not identify an element in this collection value", but the element does exist

Terraform
1

I apologize, I cannot figure out a way to provide a simpler test-case. I am certainly open to suggestions!

$ terraform version
Terraform v1.1.7
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.58.0

I’m trying to build a ‘local’ from a map, feed it through some TF Resources, and have it spit out a bunch of SQS queues, SNS topics, and create relationships between them. One of my for_each loops is erroring with

│ Error: Invalid index
│ 
│   on sns.tf line 240, in resource "aws_sns_topic_subscription" "prod_sns_subs":
│  240:   topic_arn = aws_sns_topic.prod_topics[each.value.topic].arn
│     ├────────────────
│     │ aws_sns_topic.prod_topics is object with 25 attributes
│     │ each.value.topic is "ActivityVisibilityUpdates"
│ 
│ The given key does not identify an element in this collection value.

However if I open up terraform console and view aws_sns_topic.prod_topics["ActivityVisibilityUpdates"], it is there, and it has a valid/expected .arn attribute:

> aws_sns_topic.prod_topics["ActivityVisibilityUpdates"].arn
"arn:aws:sns:us-west-2:1234:ProductionActivityVisibilityUpdates.fifo"

The map that I’m building from is:

variable "sqs_queues" {
  type = map(object({
    fifo              = bool
    highthruput       = bool
    topics            = optional(list(string))
    retention_seconds = optional(number)
  }))

  default = {
    ActivityVisibilityUpdates = {
      topics      = ["ActivityVisibilityUpdates"]
      fifo        = true
      highthruput = false
    },

The local that I’m constructing from that is:

locals {
  prod_sns_topics = merge([
    for k, v in var.sqs_queues : {
      for topic in v.topics :
      "Production${k}_Production${topic}" => {
        queue = "${k}"
        topic = "${topic}"
        fifo  = "${v.fifo}"
      }
    }
  ]...)

and the TF I’m executing, via the AWS provider, is:

resource "aws_sqs_queue" "prod_queues_dlq" {
  for_each = var.sqs_queues

  name       = each.value.fifo ? "Production${each.key}_dlq.fifo" : "Production${each.key}_dlq"
  fifo_queue = each.value.fifo

  tags = {
    Environment = "Production"
  }
}

resource "aws_sqs_queue" "prod_queues" {
  for_each = var.sqs_queues

  name                      = each.value.fifo ? "Production${each.key}.fifo" : "Production${each.key}"
  fifo_queue                = each.value.fifo
  fifo_throughput_limit     = each.value.highthruput ? "perMessageGroupId" : null
  deduplication_scope       = each.value.highthruput ? "messageGroup" : null
  message_retention_seconds = each.value.retention_seconds

  policy = data.aws_iam_policy_document.prod_sqs_default_policy[each.key].json

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.prod_queues_dlq[each.key].arn
    maxReceiveCount     = 5
  })

  depends_on = [
    aws_sqs_queue.prod_queues_dlq,
  ]

  tags = {
    Environment = "Production"
  }
}
resource "aws_sns_topic" "prod_topics" {
  for_each = local.prod_sns_topics


  name       = each.value.fifo ? "Production${each.value.topic}.fifo" : "Production${each.value.topic}"
  fifo_topic = each.value.fifo

  tags = {
    Environment = "Production"
  }
}

resource "aws_sns_topic_subscription" "prod_sns_subs" {
  for_each = local.prod_sns_topics

  topic_arn = aws_sns_topic.prod_topics[each.value.topic].arn

  protocol = "sqs"
  endpoint = aws_sqs_queue.prod_queues[each.value.queue].arn
  depends_on = [
    aws_sqs_queue.prod_queues,
    aws_sns_topic.prod_topics,
  ]

}

I cannot for the life of me figure out why its reporting the “Invalid index” error, but terraform console reports differently.

2

Hi @law,

One important difference between planning and using the console is that the console uses the resource instances recorded in your most recent state snapshot, regardless of what’s in the configuration, whereas planning works against the desired state that the configuration describes.

In your case here it seems like your for_each collection has compound keys like “ProductionSomething_ProductionSomething”, and so there would not be one named ActivityVisibilityUpdates.

The topic and topic subscription both seem to have the same for_each so when referring from topic subscription to topic it should work to use each.key as an index, because the two have the same instance keys.

1 Like