Creating dynamic blocks from list of objects

Terraform
1

I want to be able to create vault policies from a yaml configuration as follows

policies:
  policy-test-1:
    - capabilities:
        - read
        - create
      path: /foo/lala
    - capabilities:
        - create
        - patch
      path: /voo/lala
  policy-test-2:
    - capabilities:
        - update
        - delete
      path: /foo/lala

i.e. I want to provision so that each policy document may be able to have multiple statements.

var.policy_statements is the output of yamldecode of the above file.

The following approach using dynamic blocks however fails

data "vault_policy_document" "this" {

  dynamic "rule" {
    for_each = var.policy_statements

    content {
      path = rule.value.path
      capabilities = rule.value.capabilities
    }
  }



}

resource "vault_policy" "this" {
  for_each = var.policy_statements

  name   = each.key
  policy = data.vault_policy_document.this[each.key].hcl

}

│ Error: Unsupported attribute
│
│   on ../path/to/policies/main.tf  line 8, in data "vault_policy_document" "this":
│    8:       capabilities = rule.value.capabilities
│     ├────────────────
│     │ rule.value is list of object with 2 elements
│
│ Can't access attributes on a list of objects. Did you mean to access
│ attribute "capabilities" for a specific element of the list, or across all
│ elements of the list?
╵
╷
│ Error: Unsupported attribute
│
│   on ../path/to/policies/main.tf line 8, in data "vault_policy_document" "this":
│    8:       capabilities = rule.value.capabilities
│     ├────────────────
│     │ rule.value is list of object with 1 element
│
│ Can't access attributes on a list of objects. Did you mean to access
│ attribute "capabilities" for a specific element of the list, or across all
│ elements of the list?

What is the correct way to run the loop, given that each object with the policy name (e.g. policy-test-1) is an array of objects having as keys paths and capabilities?

2

I inherited a Vault setup that managed policies in YAML in the way you propose.

Based on my experience, I strongly recommend that you do not do this.

1) All policies in one file is bad

By placing all policies in one file like this, it becomes harder to easily visually see the separation between policies, and when multiple people work on the policy definitions, the possibility for Git (or other version control system) merge conflicts is created, when people change policies which just happen to be adjacent in the file.

As the number of policies all grouped together in one file increases, in a large Vault instance used by multiple teams, you can end up with a huge unwieldy YAML file. I know by costly experience that parsers get very slow when working with 1 megabyte of YAML, and GitHub’s web UI experience degrades.

Additionally, I learnt that people in general struggle to understand where to put comments on items in a large YAML list of nested objects.

2) Don’t invent your own policy dialect

From experience, again, I find it’s a disadvantage to need to train your users to understand a custom YAML dialect of Vault policy specification, when all the documentation you’ll find on the internet uses HCL.

Also, in the particular YAML representation choices made above, you make the capabilities lists very verbose, whilst the important identifiying item in each policy element, the path, is pushed to the end of the element, making the policies less easily readable than the canonical HCL forms.

3) Don’t use the Terraform vault_policy_document data source if you can avoid it

You’ll be storing all of your policies in your Terraform state file twice.

Also, if you’re using Vault Enterprise, the control_group feature is not exposed in the Terraform data source.

So, what should you do instead

Just write your policies as individual HCL files.

Use Terraform’s fileset function to get a list of all of the filenames matching *.hcl, and the file function to load the content, which you can then pass directly to your Terraform vault_policy resource.

1 Like