Terraform state file behavior

Terraform
1

We’re not sure if this is a terraform 12 thing or just something I’m simply doing wrong.
Some new code completely in terraform 12 - when we apply the state file is updated every time regardless of any changes. We have much different code in 11 (both modularized) but we don’t see this behavior there. Is this expected?

--- a/federation_iam/CloudOpsState/968600917556-terraform.tfstate
+++ b/federation_iam/CloudOpsState/968600917556-terraform.tfstate
@@ -1,7 +1,7 @@
 {
   "version": 4,
   "terraform_version": "0.12.26",
-  "serial": 79,
+  "serial": 80,
   "lineage": "af4fd9cd-e1ff-9b2a-4989-d64d09608e59",
   "outputs": {},
   "resources": [
@@ -17,7 +17,7 @@
           "attributes": {
             "account_id": "XXXXXXXXXXXXX",
             "arn": "arn:aws:sts::XXXXXXXXXXXX:assumed-role/some-role/org-script",
-            "id": "2020-06-10 15:21:42.969888429 +0000 UTC",
+            "id": "2020-06-10 15:34:49.663208607 +0000 UTC",
             "user_id": "XXXXXXXXXXXXXXX:org-script"
           }
         }
2

Hi @scoleri,

Unfortunately the summarizing done by diff here has removed the part of the state that would tell me what resource type this object has. It looks like something in the provider-side logic for that resource type is writing in an updated id and thus causing Terraform to write a new state snapshot, and if you can let me know which resource type that is I could take a look in the provider code to see what it’s doing and why it might be behaving that way.

It’d also be helpful if you could show what changes terraform plan or terraform apply are proposing to make prior to generating this modified state snapshot, if any.

3

There were no changes:

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

{
  "version": 4,
  "terraform_version": "0.12.26",
  "serial": 80,
  "lineage": "af4fd9cd-e1ff-9b2a-4989-d64d09608e59",
  "outputs": {},
  "resources": [
    {
      "module": "module.iam-federation",
      "mode": "data",
      "type": "aws_caller_identity",
      "name": "current",
      "provider": "provider.aws",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "account_id": "XXXXXXXXXXXXX",
            "arn": "arn:aws:sts::XXXXXXXXXXX:assumed-role/some-role/org-script",
            "id": "2020-06-10 15:34:49.663208607 +0000 UTC",
            "user_id": "AROA6DBI5CY2OLTJSPODY:org-script"
          }
        }
      ]
    },
    {
      "module": "module.iam-federation",
      "mode": "managed",
      "type": "aws_iam_policy",
4

Ahh, interesting… it’s a data source.

It looks like what is happening here is that Terraform is reading all of the data sources during its refresh phase (as usual) and this particular one is “unstable” in that it produces a different value each time it runs.

Somewhere along the line (I’m afraid I don’t remember in exactly which version) Terraform started accepting refresh-only changes like this as changes to be saved because it’s relatively common to have a data source result being the main effect of a module, if e.g. the resulting state will be consumed with terraform_remote_state.

However, Terraform expects data sources to keep returning the same result unless the requested remote object has actually changed. This particular data source is producing a different timestamp each time it runs, so it never converges on a fixed result and there’s always something new to save after every apply.

I think the ideal resolution for this would be for the AWS provider to populate that id attribute with something that doesn’t change on every read, such as the same string that’s in arn: that would then change only if Terraform were being run as a different principal than last time. It looks like the relevant code for this is here:

It might be worth opening an issue and/or PR in the AWS provider repository to discuss this. In particular, I’m not sure if the AWS provider team would consider this as a breaking change: technically it is a change in the formatting of one of the attributes of this data source, but in practice it’s an attribute value that wasn’t especially useful anyway.

5

Thanks for looking - i opened an issue. Please let me know if you can think of anything.