Lists with trailing commas templated by terraform, rejected by AWS

piyat · February 27, 2020, 3:49pm

Hey!

Background

To give an overview of what I’m doing - I’m working on a project in tf 0.12. The goal is to stand up some databases and data migration infrastructure. I’ve put together a couple of modules to handle that and a calling module to supply input.

In my test environment I have 3 RDS instances, 3 DMS replication instances (1 per RDS), and 6 DMS endpoints (3 source, 3 target). What I want to do is allow some database, dev types into the AWS console so they can configure the endpoints, and replication tasks manually and then once happy I can port that into the module.

Issue

I want to create a few sets of aws_iam_policy, aws_iam_group and aws_iam_group_policy_attachment resources so that per environment all users in the group will have certain permissions to explicitly defined DMS resources.

The IAM policy that is being rendered contains syntax errors, and as far as I can tell terraform is adding a trailing comma to list objects in the policy definition.

Versions
Terraform: 0.12.19
provider.aws: version = “2.46”
provider.onepassword: version = “0.5”
provider.random: version = “2.2”

What I’m doing

I’m using outputs/data sources to source 3 tuples of dms ARN’s from a separate child module - one ‘list’ each for source endpoints, target endpoints and repl instances - some example data below.
I’m merging these in a local var into a single list
I’m formatting this var with jsonencode and then attempting to use it as a templating var in a json iam policy template
- This didn’t work, so I’ve attempted the same using the data source aws_iam_policy_document, but ran into the same error
terraform console

local.all_dms_merged
[
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
]
jsonencode(local.all_dms_merged)
[“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”]

At this point, it looks as though jsonencode has done the trick and I now have a properly formatted string to inject into my template, so I run this:

resource "aws_iam_policy" "dms_policy" {
name        = "tf-dms-permissions-${terraform.workspace}"
description = "Policy allowing console users to access dms resources created in terraform workspace."
path        = "/"
policy = templatefile("${path.module}/policies/dms_permissions.json", { dms_resources = jsonencode(local.all_dms_merged) })
}

The json policy template saved in ${path.module}/policies/dms_permissions.json

{
“Version”: “2012-10-17”,
“Sid”: “DMSAllowedOperations”,
“Statement”: [
{
“Effect”: “Allow”,
“Resource”: [
${dms_resources}
],
“Action”: [
“dms:DescribeSchemas”,
“dms:DescribeRefreshSchemasStatus”,
“dms:ModifyReplicationTask”,
“dms:StartReplicationTask”,
“dms:DescribeEventSubscriptions”,
“dms:DescribeEndpointTypes”,
“dms:DescribeEventCategories”,
“dms:StartReplicationTaskAssessment”,
“dms:DescribeOrderableReplicationInstances”,
“dms:ListTagsForResource”,
“dms:DescribeConnections”,
“dms:DescribeReplicationInstances”,
“dms:DeleteReplicationTask”,
“dms:TestConnection”,
“dms:DescribeEndpoints”
]
}
]
}

Expected

The rendered policy file is formatted as specified in either the template supplied to the templatefile function or in the data.aws_iam_policy_document renderer.

Actual

The aws iam/CreatePolicy API rejects the rendered policy with: MalformedPolicyDocument

2020/02/27 11:25:38 [DEBUG] aws_iam_policy.dms_policy: apply errored, but we’re indicating that via the Error pointer rather than returning it: Error creating IAM policy example: MalformedPolicyDocument: Syntax errors in policy.

The terraform plan looks like this:

aws_iam_policy.dms_policy will be created

resource “aws_iam_policy” “dms_policy” {

arn = (known after apply)
2020/02/27 11:25:34 [DEBUG] command: asking for input: “Do you want to perform these actions in workspace "dev"?”

description = “Policy allowing console users to access dms resources created in terraform workspace.”

id = (known after apply)

name = “tf-dms-permissions-dev”

path = “/”

policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ “dms:DescribeSchemas”,
+ “dms:DescribeRefreshSchemasStatus”,
+ “dms:ModifyReplicationTask”,
+ “dms:StartReplicationTask”,
+ “dms:DescribeEventSubscriptions”,
+ “dms:DescribeEndpointTypes”,
+ “dms:DescribeEventCategories”,
+ “dms:StartReplicationTaskAssessment”,
+ “dms:DescribeOrderableReplicationInstances”,
+ “dms:ListTagsForResource”,
+ “dms:DescribeConnections”,
+ “dms:DescribeReplicationInstances”,
+ “dms:DeleteReplicationTask”,
+ “dms:TestConnection”,
+ “dms:DescribeEndpoints”,
]
+ Effect = “Allow”
+ Resource =
+ [
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:rep:somevalue”,
],
+ Sid = “DMSAllowedOperations”
},
]
+ Version = “2012-10-17”
}
)
}

The problem with this policy it seems is the trailing comma in both the Action list and Resource list.

If I drop this policy into policy sim and remove these trailing commas, the json is valid and I’d expect the call to AWS would succeed.

If this is a known bug - is there a workaround?

grimm26 · February 27, 2020, 9:13pm

You should use https://www.terraform.io/docs/providers/aws/d/iam_policy_document.html to create policies

piyat · February 28, 2020, 9:05am

Hey Mark - thanks for replying. I did mention above that I tried this too, with the same error.

grimm26 · February 28, 2020, 3:12pm

Works fine for me: https://gist.github.com/grimm26/2a2b1a285edb9d364b906cb607912fd1
I will reiterate that you should not try to build json policy from scratch. You say you tried using aws_iam_policy_document, but you should retry. Don’t use a templatefile to build a policy document.

piyat · February 28, 2020, 6:40pm

Thanks again Mark and cheers for the gist.

I agree, when you supply the resource list literally to aws_iam_policy_document, terraform templates it right and I can create the policy. No sweat.

The gist I’ve attached illustrates the problem I’m facing:

gist.github.com

https://gist.github.com/piyat/840649f821234abefa636afe3932cd10

gistfile1.txt

TEST 1: Set up an equivalent var and try to use it in data.aws_iam_policy_document.
FAILS, incorrect data type tuple not string

locals {
  a_test_var = [
          "arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue",
          "arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue",
          "arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue"
  ]
}

This file has been truncated. show original

Maybe I’ve not been clear in how I’ve asked this question - so I really appreciate your help.

The bit that I’m not understanding is how to handle supplying the local variable to any variant of terraform resource which outputs json where I need to convert that variable from a tuple to a json compliant list whilst also adhering to the terraform template schema expecting a string.

grimm26 · February 28, 2020, 7:19pm

https://gist.github.com/piyat/840649f821234abefa636afe3932cd10#file-gistfile1-txt-L33-L35 this will throw errors because local.a_test_var is already a list and you put it in which makes it a list with the first element being a list of strings.

Your second test is throwing an error because you are jsonencoding the a_test_var variable for some reason. aws_iam_policy_document doesn’t expect json - you use it to generate json. It should be:
locals {
a_test_var = [
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”,
“arn:aws:dms:eu-west-1:xxxxxxxxxxxx:endpoint:somevalue”
]
}

data "aws_iam_policy_document" "dms" {
  statement {
    sid = "DMSAllowedOperations"
    actions = [
      "dms:DescribeSchemas",
      "dms:DescribeRefreshSchemasStatus",
      "dms:ModifyReplicationTask",
      "dms:StartReplicationTask",
      "dms:DescribeEventSubscriptions",
      "dms:DescribeEndpointTypes",
      "dms:DescribeEventCategories",
      "dms:StartReplicationTaskAssessment",
      "dms:DescribeOrderableReplicationInstances",
      "dms:ListTagsForResource",
      "dms:DescribeConnections",
      "dms:DescribeReplicationInstances",
      "dms:DeleteReplicationTask",
      "dms:TestConnection",
      "dms:DescribeEndpoints",
    ]
    effect    = "Allow"
    resources = local.a_test_var
  }
}

apparentlymart · March 3, 2020, 11:08pm

Hi @piyat,

Terraform is using its own syntax to render the proposed change to the policy argument, after parsing the given value as JSON. You can tell when Terraform is doing this because it includes an extra marker that is intended to resemble a call to the jsonencode function:

    + policy = jsonencode(
        + {
              ... and then all of your policy document content ...
          },
      )

The fact that Terraform was able to produce this structural diff of the JSON document tells me that the value is valid JSON internally: Terraform managed to parse it and then format it as a diff in the plan output. The value shown there is not a byte-for-byte representation of the JSON data to be submitted, but rather a pretty-printed representation of the effective data inside that JSON document. (If you were making a change to an existing policy rather than creating a new one, Terraform would use this syntax to show you specifically which part of the data structure had changed.)

Therefore it seems that the syntax error mentioned in the error message is not a JSON syntax error, but rather an IAM policy syntax error. That is, an error within one of the values you’ve included in the JSON data, such as invalid ARN syntax or use of an incorrect JSON value type somewhere in the structure.

As @grimm26 noted, the aws_iam_policy_document data source can help here because it understands the IAM policy schema and so it can (to a certain extent) validate that you’re producing a valid IAM policy data structure and ensure that the result is valid JSON syntax.

piyat · March 11, 2020, 9:24am

Hey @grimm26, @apparentlymart. Sorry it’s taken me a while to reply. Thanks ever so much for the explanation. I had confused matters asking a question about two different means of producing Iam policy docs so appreciate you taking the time to help me understand. Indeed I was doing something silly and uneccesarily passing the list to jsonencode in data.aws_iam_policy_document.dms. All works a treat now. Thanks again

vissie · March 11, 2020, 12:02pm

I have exactly the same issue with the following code and I cant figure out how to resolve.

data "aws_iam_policy_document" "this" {
  statement {
    effect = "Allow"

    principals {
      type = "AWS"
      identifiers = [
        "arn:aws:iam::123456789012:user/test",
        "arn:aws:iam::012345678901:user/test"
      ]
    }

    actions = [
      "s3:GetObject",
      "s3:ListBucket"
    ]

    resources = [
      "arn:aws:s3:::${var.bucket_name}",
      "arn:aws:s3:::${var.bucket_name}/*"
    ]

  }
}

resource "aws_s3_bucket" "terraform_state" {
  bucket = var.bucket_name
  policy = data.aws_iam_policy_document.this.json

  versioning {
    enabled = var.bucket_versioning
  }

  tags = module.tag-label.tags
}

This produces the following plan

Terraform will perform the following actions:

  # module.s3-simple.aws_s3_bucket.terraform_state will be updated in-place
  ~ resource "aws_s3_bucket" "terraform_state" {
        acl                         = "private"
        arn                         = "arn:aws:s3:::x-terraform-state-playpen"
        bucket                      = "x-terraform-state-playpen"
        bucket_domain_name          = "x-terraform-state-playpen.s3.amazonaws.com"
        bucket_regional_domain_name = "x-terraform-state-playpen.s3.eu-west-1.amazonaws.com"
        force_destroy               = false
        hosted_zone_id              = "Z1BKCTXD74EZPE"
        id                          = "x-terraform-state-playpen"
      ~ policy                      = jsonencode(
          ~ {
              ~ Statement = [
                  ~ {
                        Action    = [
                            "s3:ListBucket",
                            "s3:GetObject",
                        ]
                        Effect    = "Allow"
                      ~ Principal = {
                          ~ AWS = "arn:aws:iam::123456789012:user/other" -> [
                              + "arn:aws:iam::123456789012:user/test",
                              + "arn:aws:iam::012345678901:user/test",
                            ]
                        }
                        Resource  = [
                            "arn:aws:s3:::x-terraform-state-playpen/*",
                            "arn:aws:s3:::x-terraform-state-playpen",
                        ]
                        Sid       = ""
                    },
                ]
                Version   = "2012-10-17"
            }
        )
        region                      = "eu-west-1"
        request_payer               = "BucketOwner"
        tags                        = {
            "Account"     = "x"
            "Application" = "x"
            "Name"        = "x"
            "Namespace"   = "x"
            "Stage"       = "x"
        }

        versioning {
            enabled    = true
            mfa_delete = false
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

and I get the same error

Error: Error putting S3 policy: MalformedPolicy: Invalid principal in policy
        status code: 400, request id: 72061323F3B9021X, host id: CNTnFYvvVdk1/naGe+RA0jrr0GJFXqgGD+3xLtLOq+oNavZzLm6c4tV3Q5sQNVkzpMLEzGTnn9E=

Versions:

Terraform v0.12.23
+ provider.aws v2.52.0

vissie · March 12, 2020, 9:14am

okay, seems like running it in debug mode it was fine, they seem to check if the iam user actually exists before they allow it to be applied where in the past it wasn’t like that.

Topic		Replies	Views
Templatefile() function doesn't work with AWS IAM AWS	0	678	November 21, 2019
Terraform malformed policy Terraform	2	5960	July 10, 2020
How to handle trailing commas in template loops Terraform	10	4391	November 1, 2022
Using formatlist() keeps complaining about template interpolation value Terraform	2	1008	January 3, 2020
Concat 2 lists within AWS IAM role policy statement Terraform	2	2713	December 17, 2019

Lists with trailing commas templated by terraform, rejected by AWS

aws_iam_policy.dms_policy will be created

Related topics