Need help with nested JSON file using locals block

Terraform
1

Hi

I’m trying to use existing AWS security group rules from JSON file and update them back. In this regarding below is the sample data

[
    {
        "Description": "dev_sg",
        "GroupName": "dev_sg",
        "IpPermissions": [
            {
                "FromPort": 80,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.1.10/32",
                        "Description": "Test office"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 80,
                "UserIdGroupPairs": []
            },
            {
                "FromPort": 0,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.10.12/32"
                    },
                    {
                        "CidrIp": "192.168.11.12/32"
                    },
                    {
                        "CidrIp": "192.168.13.12/32",
                        "Description": "Test office"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 6555,
                "UserIdGroupPairs": []
            },
            {
                "IpProtocol": "-1",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.13.11/32"
                    },
                    {
                        "CidrIp": "192.168.13.14/32"
                    },
                    {
                        "CidrIp": "192.168.13.12/32",
                        "Description": "Test office new ISP"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "UserIdGroupPairs": []
            },
            {
                "FromPort": 22,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.13.16/32"
                    },
                    {
                        "CidrIp": "192.168.13.38/32"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 22,
                "UserIdGroupPairs": []
            },
            {
                "FromPort": 443,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.1.10/32",
                        "Description": "Test office"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 443,
                "UserIdGroupPairs": []
            },
            {
                "FromPort": 8433,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.13.178/32",
                        "Description": "testing2"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 8433,
                "UserIdGroupPairs": []
            }
        ],
        "OwnerId": "2542495439859",
        "GroupId": "sg-25y495lngkelfgm",
        "IpPermissionsEgress": [
            {
                "IpProtocol": "-1",
                "IpRanges": [
                    {
                        "CidrIp": "0.0.0.0/0"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "UserIdGroupPairs": []
            }
        ],
        "VpcId": "vpc-sldngslgw454232624"
    },
    {
        "Description": "SG rule",
        "GroupName": "allow_access",
        "IpPermissions": [
            {
                "FromPort": 6379,
                "IpProtocol": "tcp",
                "IpRanges": [
                    {
                        "CidrIp": "192.168.13.34/32",
                        "Description": "test data"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "ToPort": 6379,
                "UserIdGroupPairs": []
            }
        ],
        "OwnerId": "2542495439859",
        "GroupId": "sg-3254365erlgkh",
        "IpPermissionsEgress": [
            {
                "IpProtocol": "-1",
                "IpRanges": [
                    {
                        "CidrIp": "0.0.0.0/0"
                    }
                ],
                "Ipv6Ranges": [],
                "PrefixListIds": [],
                "UserIdGroupPairs": []
            }
        ],
        "VpcId": "vpc-sldngslgw454232624"
    }
]

I see errors while trying to use in through locals block. Below is the code

locals {
  config = jsondecode(file("test-json.json"))

  nested_details = flatten([
    for k, v in local.config : [
      for keys, values in v : [

      ]
    ]
  ])
}

output "result" {
  value = local.config
}

To understand on more of it, I tried to use terraform console with the command flatten([for k,v in local.config : [for key,values in v: [ for value in values: value.IpPermissions ] ] ]) but it throws an error.

 Error: Unsupported attribute
│
│   on <console-input> line 1:
│   (source code not available)
│
│ This object does not have an attribute named "IpPermissions".

I’m trying to access FromPort, Toport , CidrIp and Description. Please help me understand how to access them to use in security_group_rule resource :pray:

2

I’m able to access the value little further with flatten([for values in local.config : [ for ipv in values["IpPermissions"] : "${ipv}" ] ])

This provided information of the IpPermissions values, still digging in :dagger:

3

It seems like you are experiencing a very common confusing situation in Terraform - the need to loop over two nested levels of data to produce resources. Here’s a previous time I answered this: How to deal with nested for_each loops in dependent ressources - #2 by maxb

There are many slight variations in how this can be written, but fundamentally you always need:

  • To loop over the outer collection:
[for security_group in local.config :
  • To loop over the inner data:
[for rule in security_group.IpPermissions :
  • To assemble an object that contains information from both levels of loop:
{
  security_group = security_group
  rule           = rule
}
  • To wrap all of the code above in flatten()
  • To loop over the flattened data again, picking out a string value that identifies each item, and will be used as the resource name:
{ for item in flatten( ... ) : "${some kind of expression}" => item }

That last one is particularly hard in this case, as security group rules don’t have a simple field works well to use as their name.

You could just decide to use their numeric index within the input JSON array, but that exposes you to Terraform potentially adding/deleting/modifying more than it needs to on future updates, if additions/deletions cause other rules to shift position in the array.

You could jsonencode the entire input element from IpPermissions, but that would lead to some particularly long and unreadable resource keys.

Or, you could write a custom expression that attempts to capture just enough of the fields to ensure uniqueness.

Anyway, putting the previous parts together:

resource "aws_security_group_rule" "something" {
  for_each = { for item in flatten(
    [for security_group in local.config :
      [for rule in security_group.IpPermissions :
        {
          security_group = security_group
          rule           = rule
    }]]
  ) : "${some_kind_of_expression}" => item }
}
4

thank you @maxb for the reply and helping me understand. I updated the code but not able to understand what values will be accessible using each.value . Is there anyway I can check on that part :pray:

Moreover the below code throws error as FromPort attribute does not exist :cry:

resource "random_pet" "example" {
  for_each = { for item in flatten(
    [for security_group in local.config :
      [for ippermissions in security_group.IpPermissions :
        [for ipranges in ippermissions.IpRanges :
          {
            security_group = security_group
            ippermissions  = ippermissions
            ipranges       = ipranges
          }
        ]
      ]
    ]
    ) : item.security_group.GroupName => item...
  }
  prefix = each.value.ippermissions.FromPort
}