Terraform 0.13.7 each.value is not working s3 bucket multiple notifications

Terraform
1

Hello,
I tried to create for_each loop for AWS s3 bucket notification.

I got these errors:

  on triggers.tf line 8, in resource "aws_s3_bucket_notification" "aws-lambda-trigger":
   8:     events              = ["${each.value.events}"]
    |----------------
    | each.value is tuple with 1 element
This value does not have any attributes.
Error: Unsupported attribute
  on triggers.tf line 9, in resource "aws_s3_bucket_notification" "aws-lambda-trigger":
   9:     filter_prefix       = "${each.value.filter_prefix}"
    |----------------
    | each.value is tuple with 1 element
This value does not have any attributes.
Error: Unsupported attribute
  on triggers.tf line 10, in resource "aws_s3_bucket_notification" "aws-lambda-trigger":
  10:     filter_suffix       = "${each.value.filter_suffix}"
    |----------------
    | each.value is tuple with 1 element
This value does not have any attributes.

I have in tfvars:

s3_triggers = [
  {
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/team/",
  filter_suffix     = ".json"  
  }
]

And in terraform I have triggers.tf

resource "aws_s3_bucket_notification" "aws-lambda-trigger" {
  for_each = {for item in var.s3_triggers:  item.create_s3_trigger => item... if item.create_s3_trigger}
  bucket   = "aws_s3_bucket.bucket[each.key].id"


  lambda_function {
    lambda_function_arn = "arn:aws:lambda:${var.region}:${var.arn_project}:function:${var.lambda_function_name}-${var.irn}-${var.sia}-${var.env}"
    events              = ["${each.value.events}"]
    filter_prefix       = "${each.value.filter_prefix}"
    filter_suffix       = "${each.value.filter_suffix}"
  }
}

Could someone help me, please?

Thanks in advance.

3

Please could you fix the very messed up formatting in your message?

This line of your config does not make sense:

  for_each = {for item in var.s3_triggers:  item.create_s3_trigger => item... if item.create_s3_trigger}

It will evaluate to:

{
  "true" = [
    {
      "create_s3_trigger" = true
      "events" = "s3:ObjectCreated:*"
      "filter_prefix" = "/input/team/"
      "filter_suffix" = ".json"
    },
  ]
}
4

I am sorry @maxb. I fixed formatting.

For now I have in tfvars this:

create_s3 = [
  {
  create            = true,
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/appointment/",
  filter_suffix     = ".json"
  },
  {
  create            = true,
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/operator/",
  filter_suffix     = ".json"
  },
  {
  create            = true,
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/sa/",
  filter_suffix     = ".json"
  }, 
  { 
  create            = true,
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/team/",
  filter_suffix     = ".json"
  }
]

and in terraform:

resource "aws_s3_bucket_notification" "aws-lambda-trigger" {
  for_each = {for item in var.create_s3:  item.filter_prefix => item... if item.create_s3_trigger}
  bucket   = "aws_s3_bucket.bucket[each.key].id"


  lambda_function {
    lambda_function_arn = "arn:aws:lambda:${var.region}:${var.arn_project}:function:${var.lambda_function_name}-${var.irn}-${var.sia}-${var.env}"
    events              = ["${each.value.events}"]
    filter_prefix       = "${each.value.filter_prefix}"
    filter_suffix       = "${each.value.filter_suffix}"
  }
}

and still is not working :frowning: Same issue. Could you help me, please?

5

I’m not sure why the ... is there. I don’t think it should be.

6

This must to be there because I am using same name for buket_name. Without … it gives error. I learned it today :slight_smile:

7

I’m not sure what you mean there?

I did also notice that the bucket line is likely to be wrong - you are setting it to a string, so that could just be a typo in the post (remove the quotes) but even then it might still be wrong. each.key would be the filter_prefix value. I’m guessing you actually want each.value.bucket_name

8

@stuart-c If I remove this I get this this error, but it looks like loop is working.

Error: Duplicate object key
  on triggers.tf line 2, in resource "aws_s3_bucket_notification" "aws-lambda-trigger":
   2:   for_each = {for item in var.create_s3: "${item.events} ${item.bucket_name}" => item if item.create_s3_trigger}
    |----------------
    | item.bucket_name is "dev-ras-initialization-dev"
    | item.events is "s3:ObjectCreated:*"
Two different items produced the key "s3:ObjectCreated:*
dev-ras-initialization-dev" in this 'for' expression. If duplicates are
expected, use the ellipsis (...) after the value expression to enable grouping
by key.
9

You need to ensure the key in the map you are producing is unique.

So you need to figure out what combination of the values need combining to achieve that. What that is saying is that using the bucket name and events isn’t sufficient. You would need to choose a different combination.

Alternatively a simpler option would be to change the create_s3 variable from being a list of maps to a map of maps. That way you could set a unique key yourself instead of having to try to programatically produce one.

10

Thank you very much @stuart-c for help. I really appreciate it.

For now I added name like this and it is working :slight_smile: :

create_s3 = [
  {
  create            = true,
  name              = "a",
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/appointment/",
  filter_suffix     = ".json"
  },
  {
  create            = true,
  name              = "b",
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/operator/",
  filter_suffix     = ".json"
  },
  {
  create            = true,
  name              = "c",
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/sa/",
  filter_suffix     = ".json"
  }, 
  { 
  create            = true,
  name              = "d",
  bucket_name       = "dev-ras-initialization-dev",
  create_s3_trigger = true,
  events            = "s3:ObjectCreated:*",
  filter_prefix     = "/input/team/",
  filter_suffix     = ".json"
  }
]
11

I will rewrited soon :slight_smile:

12

Why are you using a list? The order doesn’t matter and having your new name field as the key in a map of maps would make more sense.

13

I wanted to deploy it as soon as possible but now it wants to deploy 32 new stuff, so I have one bucket and it wants to create multiple buckets, policies…:frowning:

14

I don’t know how for_each will look like with maps of map.

15

Ok, so here is the solution, if someone else will look for it.

tfvars

create_s3_notifications = true
s3_notifications = {
    bucket = {
        name            = "dev-ras-initialization-dev"
        filters = [
            {
                events          = "s3:ObjectCreated:*"
                name            = "filter1"
                filter_prefix   = "/input/appointment/"
                filter_suffix   = ".json"
            },
            {
                events          = "s3:ObjectCreated:*"
                name            = "filter2"
                filter_prefix   = "/input/operator/"
                filter_suffix   = ".json"
            },
            {
                events          = "s3:ObjectCreated:*"
                name            = "filter3"
                filter_prefix   = "/input/sa/"
                filter_suffix   = ".json"
            },
            {
                events          = "s3:ObjectCreated:*"
                name            = "filter4"
                filter_prefix   = "/input/team/"
                filter_suffix   = ".json"
            }
        ]
    }
}

terraform

resource "aws_lambda_permission" "allow_bucket" {
  count         = var.create_s3_notifications ? 1 : 0
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = "${var.lambda_function_name}-${var.irn}-${var.sia}-${var.env}"
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.bucket[count.index].arn
}

resource "aws_s3_bucket_notification" "aws-lambda-trigger" {
  for_each                         = {for item in var.s3_notifications:  item.name => item if var.create_s3_notifications}
  bucket                           = each.value.name

  dynamic "lambda_function" {
    for_each = [for item in each.value.filters: {
      suffix = item.filter_suffix
      prefix = item.filter_prefix
      events = item.events
    }]

    content {
      lambda_function_arn = "arn:aws:lambda:${var.region}:${var.arn_project}:function:${var.lambda_function_name}-${var.irn}-${var.sia}-${var.env}"
      events              = [lambda_function.value.events]
      filter_prefix       = lambda_function.value.prefix
      filter_suffix       = lambda_function.value.suffix
    }
  }
}