For_each : tuple with a dynamic value

Hello,
I’m trying to append a security group to a list(tuple?) and TF is not happy about it.

locals {
  sec_grp0 = tolist(var.launchtempl.security_group_names)
  sec_grp  = concat(local.sec_grp0, data.tfe_outputs.alb.values.alb_security_group.name)
 }

data "aws_security_group" "sg-name" {
  for_each = local.sec_grp
  name     = each.key
}

output:

data.tfe_outputs.alb: Reading...
data.aws_subnet.subnet-id["private-app-1"]: Reading...
data.aws_subnet.subnet-id["private-app-2"]: Reading...
data.tfe_outputs.alb: Read complete after 1s [id=XXXXXX]
data.aws_subnet.subnet-id["private-app-2"]: Read complete after 0s [id=subnet-01d50267c0b6e5xxx]
data.aws_subnet.subnet-id["private-app-1"]: Read complete after 0s [id=subnet-0e5d76c5e9b285xxx]
data.aws_ami.AM2: Read complete after 0s [id=ami-0fd9d2af2c98xxx]
╷
│ Error: Invalid function argument
│ 
│   on autoscale.tf line 12, in locals:
│   12:   sec_grp  = concat(local.sec_grp0, data.tfe_outputs.alb.values.alb_security_group.name)
│     ├────────────────
│     │ while calling concat(seqs...)
│ 
│ Invalid value for "seqs" parameter: all arguments must be lists or tuples; got string.

The variables are:

launchtempl = {
...
  security_group_names = [
    "sg_NonProd",
    "sg_NonProd_RDS",
  ]
...
}

The console reveals:

> var.launchtempl.security_group_names
[
  "sg_NonProd",
  "sg_NonProd_RDS",
]
> type(var.launchtempl.security_group_names)
tuple([
    string,
    string,
])
> data.tfe_outputs.alb.values.alb_security_group.name
(known after apply)
> type(data.tfe_outputs.alb.values.alb_security_group.name)
dynamic

> type(local.sec_grp0)
list(string)

Thank you for looking.

I suspect this error is due to the tfe_output value been “unknown” at the time of execution…

and probably this Unknown values should not block successful planning · Issue #30937 · hashicorp/terraform · GitHub ticket is relevent by @apparentlymart

Hi @AZZ,

I think this problem would be resolved by changing the second argument of concat to be a tuple expression with a single element, like this:

concat(local.sec_grp0, [data.tfe_outputs.alb.values.alb_security_group.name])

Your terraform console experiments yielded dynamic as the type for this attribute because this configuration has not yet been applied and so Terraform has not yet dealt with reading that data source, and workspace output values can be of any type so Terraform cannot predict the type ahead of time.

However, I’m inferring from the error message that Terraform found a string in that output value after reading the tfe_outputs data source, and so wrapping that reference in brackets would create a value of type tuple([string]), which is an acceptable argument type for the concat function.


Bonus chatter: In Terraform v1.7.0, released just yesterday, terraform console has a new option -plan which causes it to do the same work as terraform plan would do before showing the console prompt, and then evaluates expressions against the planned new state, rather than against the prior state.

This does come at the cost of a slower startup time due to Terraform needing to refresh all existing objects and read data sources before showing the prompt, but in return in this case it would’ve been able to give you some more useful feedback:

> data.tfe_outputs.alb.values.alb_security_group.name
"whatever-the-name-actually-is"
> type(data.tfe_outputs.alb.values.alb_security_group.name)
string

Of course that doesn’t help in retrospect since you were working on this with an older version of Terraform, but I mention it just in case it’s useful for someone debugging a similar problem in future.

1 Like

@apparentlymart ,
in the reverse order of your reply:

  1. TF 1.17 - this sounds so useful. But:
 wget -O- https://rpm.releases.hashicorp.com/$release/hashicorp.repo | sudo tee /etc/yum.repos.d/hashicorp.repo
--2024-01-18 13:42:47--  https://rpm.releases.hashicorp.com//hashicorp.repo
Resolving rpm.releases.hashicorp.com (rpm.releases.hashicorp.com)... 108.138.128.125, 108.138.128.70, 108.138.128.129, ...
Connecting to rpm.releases.hashicorp.com (rpm.releases.hashicorp.com)|108.138.128.125|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-01-18 13:42:47 ERROR 404: Not Found.

dnf update fails with the same issue.

It appears the “fedora” is not hosted on the site?

Hi @AZZ,

I’m afraid I’m not really involved in how that RPM repository gets constructed, so I’m not sure what’s going on there. However, it seems like you’ve run a command containing a $release substitution while that variable isn’t defined in your shell, and so the argument expanded as:

https://rpm.releases.hashicorp.com//hashicorp.repo

Based on the instructions on the download page, it seems like that path element should be fedora for your case, so you could either set that variable in your shell or just replace $release with a hard-coded fedora and then it seems to resolve for me:

$ curl https://rpm.releases.hashicorp.com/fedora/hashicorp.repo
[hashicorp]
name=Hashicorp Stable - $basearch
baseurl=https://rpm.releases.hashicorp.com/fedora/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://rpm.releases.hashicorp.com/gpg

[hashicorp-test]
name=Hashicorp Test - $basearch
baseurl=https://rpm.releases.hashicorp.com/fedora/$releasever/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://rpm.releases.hashicorp.com/gpg

(This does assume that your system’s “releasever” and “basearch” are also ones supported by the HashiCorp repository, but that is at least handled by the RPM tools itself, and so doesn’t need to be manually configured in your shell.)

As for the concat():
the suggestion worked, but “for_each” is not happy. To be frank, the issue with “for_each” is what I am after:

locals {
sec_grp0 = tolist(var.launchtempl.security_group_names)
sec_grp  = concat(local.sec_grp0, [data.tfe_outputs.alb.values.alb_security_group.name])
}

data "aws_security_group" "sg-name" {
for_each = local.sec_grp
name = each.key
}

output:

Error: Invalid for_each argument
│ 
│   on autoscale.tf line 19, in data "aws_security_group" "sg-name":
│   19:   for_each = local.sec_grp
│     ├────────────────
│     │ local.sec_grp is tuple with 3 elements
│ 
│ The given "for_each" argument value is unsuitable: the "for_each" argument must be a map, or set of strings, and you have provided a value of type tuple.
╵

Adding toset():

data "aws_security_group" "sg-name" {
for_each = local.sec_grp
name = each.key
}

output:

│ Error: Invalid for_each argument
│ 
│   on autoscale.tf line 19, in data "aws_security_group" "sg-name":
│   19:   for_each = toset(local.sec_grp)
│     ├────────────────
│     │ local.sec_grp is tuple with 3 elements
│ 
│ Sensitive values, or values derived from sensitive values, cannot be used as for_each arguments. If used, the sensitive value could be exposed as a resource
│ instance key.
╵

now the access to the repo is working . Must be my bad timing. Fedora 37 is nolonger supported. This makes sense of course. I’ll upgrade the OS later this week and let you know on 17th version.

REF: Official Packaging Guide

If you do really consider your security group names as sensitive then it isn’t really clear how to proceed here because sensitive values cannot be instance keys (as the error message suggests) but there’s nothing else available to use as a proxy for those here.

However, I suspect that this is just the hashicorp/tfe provider being overly cautious and marking the entire values object as sensitive “just in case”, and if that’s true – i.e. if you disagree with Terraform Cloud’s provider that this should be treated as sensitive – then you can explicitly remove the sensitive marking to make this work:

concat(
  local.sec_grp0,
  [nonsensitive(data.tfe_outputs.alb.values.alb_security_group.name)],
)

If you use this approach then of course you’ll need to make sure that a sensitive value never actually appears in that attribute, but it seems unlikely to me that a security group name would ever be really considered “sensitive” in the Terraform sense.

And that did the trick!

I had nonsensitive() added last night and ended up with the code still not working… Well, now it does work. :slight_smile:
I’ll need to understand what i did wrong. thank you very much @apparentlymart

P.S. I bit the bullet and upgrading the OS now :man_facepalming: