Terraform 0.14: concise plan diff feedback thread

[alisdair]

We have a new concise diff renderer, released today in Terraform 0.14.0-alpha20200910. This post explains why we’ve taken this approach, how the rendering algorithm works, and asks for your feedback.

You can try out this feature today:

• Download Terraform 0.14.0-alpha20200910
• Review the changelog
• More on Terraform 0.14-alpha release

Background

Terraform 0.12 introduced a new plan file format and structural diff renderer, which was a significant change from 0.11. Most notably, for updated resources, Terraform moved from showing only the changed attribute path/values, to showing the entire resource with changed values prefixed with ~.

For plans which slightly change existing resources, this can result in very large diffs which make it difficult to find the actual changes. Several GitHub issues report this as a problem: 23402, 21639, and 25348. There is also a community pull request which implements a -concise flag to render only the changed attributes.

The Terraform core team has not been able to engage with the community on this problem due to working on other projects. This is now a priority for us, and after assessing the existing community feedback we have an experimental concise diff renderer to share with you. Our hope is that this approach will meet the following requirements:

1. Users can more quickly read the plan output to see the changes for each resource. This means reducing the diff so that we don’t show all of the fields.

2. Resources are still identifiable in this shorter diff view. We should display essential identifying fields such as id, so that it is clear which resources are affected.

3. Terraform has one diff rendering format, not full and concise options. Adding command-line switches puts the burden on users to discover the feature, and makes maintenance more difficult.

Diff Renderer Design

The diff renderer used by terraform plan, terraform apply, and terraform show <planfile> will be updated to hide unchanged and irrelevant fields. This means:

• Always show all identifying fields, initially defined as id, name, and tags, even if unchanged
• Only show changed, added, or removed primitive values: string, number, or bool
• Only show added or removed elements in unordered collections and structural types: map, set, and object
• Show added or removed elements with up to two contextual unchanged elements for sequence types: list and tuple
• Only show added or removed nested blocks, or blocks with changed attributes

If any attributes, collection elements, or blocks are hidden, a count will be kept and displayed at the end of the parent scope. This ensures that the diff is clearly only displaying a subset of the resource.

Diff Rendering Details

Changed primitive values will render as before: prefixed with a ~, and showing the old and new value. Any hidden attributes will be counted, and the total displayed at the end of the resource:

~ attribute_name   = "old value" -> "new value"
  # (12 unchanged attributes hidden)

Unordered collections (sets or maps) with added or removed elements will use the same approach:

~ regions = [
    - "us-east-1",
    - "us-west-2",
    + "eu-west-1",
    + "eu-west-2",
      # (1 unchanged element hidden)
  ]

Sequences (lists or tuples) will display context around changes:

~ words     = [
      # (3 unchanged elements hidden)
    - "delta",
    + "data",
      "echo",
      # (5 unchanged elements hidden)
      "kilo",
    - "Lima",
    + "London",
      "Mike",
      " (13 unchanged elements hidden)
  ]

Any nested blocks for a resource which are unchanged will also be counted and the total displayed:

# (3 unchanged blocks hidden)

When color is enabled, these hidden value counts will be displayed in a lower-contrast color.

Diff Examples

Changes to attributes

  # test_resource.foo will be updated in-place
  ~ resource "test_resource" "foo" {
        id       = "foo_123"
      ~ checksum = 28987129 -> (known after apply)
      - mode     = "test" -> null
        name     = "Foo Test"
        tags     = []
      ~ totals   = {
          - "bar" = 5 -> null
          + "baz" = 5
            # (2 unchanged elements hidden)
        }
      ~ values   = [
          - "alpha",
          - "gamma",
          + "alpaca",
          + "goblin",
          + "zephyr",
            # (23 unchanged elements hidden)
        ]
        # (5 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

Adding a new block

  # test_resource.foo will be updated in-place
  ~ resource "test_resource" "foo" {
        id       = "foo_123"
      ~ checksum = 28987129 -> (known after apply)
        name     = "Foo Test"
        tags     = []
        # (8 unchanged attributes hidden)

      + sub_item {
          + identifier = "bar"
          + size       = 4
          + type       = "whatever"
        }
        # (3 unchanged blocks hidden)
    }

Example from issue 23402

  # google_container_cluster.nonprod will be updated in-place
  ~ resource "google_container_cluster" "nonprod" {
      ~ additional_zones            = [
          - "us-central1-a",
          - "us-central1-b",
          - "us-central1-c",
        ]
        id                          = "nonprod"
        name                        = "nonprod"
      - region                      = "us-central1" -> null
        # (24 unchanged attributes hidden)

        # (13 unchanged blocks hidden)
    }

Prototype Screenshots

Changed attributes:

Changed set elements:

Changed list elements:

Block attributes changed:

New block added:

Next Steps

We have an implementation of this algorithm that we are releasing today in a Terraform prerelease build. We’ll gather and respond to feedback from this early release, and if the process goes well release the new concise diff feature as part of Terraform 0.14.0.

Please share your thoughts!

Alisdair McDiarmid, Terraform Core Team

[vasylenko]

That looks nice, indeed. For example, when I change a CloudFront distribution now (0.13.x), I “must read” all the attributes in the plan output but I don’t need/want this.
So is it going to be an optional feature enabled with -concise flag?

[alisdair]

@vasylenko

So is it going to be an optional feature enabled with -concise flag?

We intend this to be the new behaviour of the plan diff, and there will not be a command-line switch.

For 0.14.0, our plan is to ship this as an experiment, but enabled by default. The experiment can be disabled by setting a TF_X_CONCISE_DIFF environment variable to 0.

[kmfk]

How does this affect or change the JSON output from a show?

[alisdair]

@kmfk There’s no change to the JSON output as a result of this work. It’s purely a text UI change.

[kmfk]

Fantastic. Also, I did realize literally as I hit submit on that question that I should have just downloaded 0.14 alpha and tried it. Thanks for following up!

[olenm]

Just getting around to TF 0.14 the last few weeks; the concise plan is super great, a lot less noise/spam - however with that being said, I feel that TF_X_CONCISE_DIFF could be documented within the terraform plan --help text display, and hopefully this option is not removed in the future. There are a number of times where I prefer to see the full list of items where a minor sub-element is changed.