Create list of maps from tuples

Terraform
1

I am stuck here please provide help.

What I want is to generate a local variable block like below

locals {
  test_ips = {
    amqp = { port = 5671, ip = "10.40.x.x”}
    https = { port = 443,   ip = "10.40.x.x”}
  }
}

Which then gets passed into the resource block

resource "aws_lb_target_group_attachment" "amq" {
  for_each    = local.test_ips
  target_group_arn  = aws_lb_target_group.amq[each.key].arn
  target_id         = each.value.ip
  port              = each.value.port
}

this is the actual code block used to generate the test_ips

 test_ips = flatten([for key, value in var.nlb_ports : {
      for ip in data.dns_a_record_set.amq.addrs: key => {ip = ip, port = value }}
])

inputs

variable "nlb_ports" {
  type    = map(number)
  default = {
    amqp  = 5671
    https = 443
  }
}

outputs

  + test_ips  = [
      + [
          + {
              + amqp = {
                  + ip   = "10.40.x.x”
                  + port = 5671
                }
            },
          + {
              + https = {
                  + ip   = "10.40.x.x”
                  + port = 443
                }
            },
        ],
    ]

But now the resource block fails with

│   on ../nlb.tf line 47, in resource "aws_lb_target_group_attachment" "amq":
│   47:   for_each    = local.test_ips
│     ├────────────────
│     │ local.test_ips is tuple with 2 elements
│ 
│ The given "for_each" argument value is unsuitable: the "for_each" argument must be a map, or set of strings, and you have provided a value of type tuple.

How can I make the list of tuples into list of maps for the foreach to work ?

2

I don’t understand what you are trying to do here, because…

  • Your target data structure shape is only capable of accomodating one IP address per port:
  • But your current expression to generate it is iterating over a datasource that can return multiple IPs:

I guess perhaps you are assuming that the DNS record set will always contain exactly one IP?

In case that’s true, then you want something more like:

test_ips = {
  for key, value in var.nlb_ports : key =>
  { port = value, ip = one(data.dns_a_record_set.amq.addrs) }
}

The attempt to use flatten here is a step in the wrong direction.

3

you are right. flatten was not the right choice. below snippet actually did the trick for an existing deployment and when the when the collection is expected to have just one ip. I am yet to test the multi ip deployment :slight_smile:

locals {
  test_ips = merge([for key, value in var.nlb_ports : {
      for ip in data.dns_a_record_set.amq.addrs: key => {ip = ip, port = value }}
]...)
}

but when i tested for a new deployment it fails :frowning: any idea how to handle this ?

│ Error: Invalid for_each argument
│ 
│   on ../nlb.tf line 53, in resource "aws_lb_target_group_attachment" "amq":
│   53:   for_each    = local.test_ips
│     ├────────────────
│     │ local.test_ips will be known only after apply
│ 
│ The "for_each" map includes keys derived from resource attributes that cannot be determined until apply, and so Terraform cannot determine the full set of keys that
│ will identify the instances of this resource.
│ 
│ When working with unknown values in for_each, it's better to define the map keys statically in your configuration and place apply-time results only in the map
│ values.
│ 
│ Alternatively, you could use the -target planning option to first apply only the resources that the for_each value depends on, and then apply a second time to fully
│ converge.
4

I don’t understand what you’re trying to do with merge here… to me it feels like you’ve just come up with a more complicated circuitous version of what I wrote, with roughly equivalent behaviour.

It’ll fail when there are multiple IPs, the same as mine will, but in a less obvious way (key conflicts in the for expression, instead of explicit use of one()).

Like the error message is pointing out, for_each keys must be be strings that can be known at plan time, not ones that are unknown until apply.

You’d have to show us more of your configuration if you want help on understanding why what you currently have doesn’t work. My guess is you’re passing something in var.nlb_ports that isn’t known until apply.

5

Apologies for the delayed response. Currently, I am utilizing a targeted plan and apply to make progress. However, I would greatly appreciate your input on achieving this task using a single plan and apply approach. My objective is to register the private IP addresses of the AWS MQ clusters to an IP-based NLB target groups. These addresses are only obtainable after the cluster has been provisioned. Please find below an explanation of my current method.

lookup the ip address by the amq_console_url

"dns_a_record_set" "amq" {
  host = regex("https://(.+)", var.amq_console_url)[0]
}

Construct locally amqp and https ip collection

  amqp_ips = [for ip in data.dns_a_record_set.amq.addrs : {
    ip_addr = ip
    port    = 5671
    proto   = "amqp"
  }]
  https_ips = [for ip in data.dns_a_record_set.amq.addrs : {
    ip_addr = ip
    port    = 443
    proto   = "https"
  }]

doing a for_each on the collection to generate the target group attachments

resource "aws_lb_target_group_attachment" "amqp" {
  for_each         = { for item in local.amqp_ips : item.ip_addr => item }
  target_group_arn = aws_lb_target_group.amq[each.value.proto].arn
  target_id        = each.value.ip_addr
  port             = each.value.port
}

resource "aws_lb_target_group_attachment" "https" {
  for_each         = { for item in local.https_ips : item.ip_addr => item }
  target_group_arn = aws_lb_target_group.amq[each.value.proto].arn
  target_id        = each.value.ip_addr
  port             = each.value.port
}

now the issue as I understand is due to me using a ip_address as the key item which is only known after the apply. I don’t have any other unique item in my collection other than the ip addresses. How else can I do this ? I tried the count approach too but failed with a similar error.

resource "aws_lb_target_group_attachment" "amqp" {
  count            = length(data.dns_a_record_set.amq.addrs)
  target_group_arn = aws_lb_target_group.amq["amqp"].arn
  target_id        = data.dns_a_record_set.amq.addrs[count.index]
  port             = 5671
}
6

Fundamentally, the architecture of Terraform means that it has a rigid plan-then-apply separation.

Resource addresses - which includes the indexes used in count or for_each - must be knowable at plan time.

Your exact IP addresses apparently can’t… but perhaps they don’t need to be. The count/for_each index doesn’t need to be the address itself - just some set of IDs that establishes how many IP addresses there will be, and can be consistently mapped to them. Why are there multiple IPs? Can you predict at plan time, how many of them there will be? Can you identify them with a counter? Or, can you give them placeholder names? (e.g. the red, yellow, blue IPs)

The key is to come up with some placeholder Terraform can know at plan time, and use as an identifier for its resource representation - and the actual IP addresses can be referenced only at apply time.

7

Thanks a million @maxb . That worked like a charm. :tada: Thanks again…