Prevent re-create existing resources when incoming data changed

Terraform
1

Hi All,

I manage AWS Codepipeline and the related resources by Terraform. In our product, Terraform generates pipeline for every branches, where the branches are produced by a bash script using terraform/external datasource.

When I create / delete a branch, and its name doesn’t cause to be at the end of the list (in alphabetical order in a json string array), Terraform console will report, that every resources will be destroyed and created again affected by branch names standing after the newly created / removed one.

Suppose, you have 4 branches and delete one:

["A", "C", "D", "E"] -> ["A", "D", "E"]

In this case, Terraform change summary reports something like 3 resources will be deleted and 2 added.

Below some relevant part from the code:

The datasource:

data "external" "git_remote_branches" {
  program = ["/bin/bash", "list_remote_branches.sh"]
}

The script:

#!/bin/bash

echo -n "{\"branches\":\""
git ls-remote --heads --quiet | while IFS= read -r line; do
    parts=($line)
    echo -n ${parts[1]#refs/heads/}","
done

echo "\"}"

Last line of the change

Plan: 1140 to add, 17 to change, 1199 to destroy.

diff.txt (3.5 MB)

This behavior is very annoying and harmful, because

  • operations take much more time
  • cannot oversee the changes
  • sometimes pipeline gets triggered so execution time should be paid

Can anybody clarify, that

  • I did something wrong
  • Is it a Terraform issue
  • …etc

If you need additional information, please let me know

2

If you are using a list this would be expected as the order and position of elements matters.

Presumably you are using count to loop through the list to create resources.

Instead it would be recommended to switch to using a map with for_each, which can usually be done via a for expression.

3

Thank you, it seems, that do what I expect