Compare two terraform state files to check imports are working

Terraform
1

We run CI pipelines in which we deploy all our infrastructure using terraform, run some tests, then destroy that infrastructure.
We also have a script which runs terraform import upon that infrastructure. I want to run some tests in our CI pipeline to check that those import scripts can successfully get us back to a true state. My desired method of doing this is to, in our CI pipeline:

  1. deploy the infrastructure
  2. rename the terraform state file to terraform.tfstate.old
  3. run the terraform import script
  4. Compare the new terraform.tfstate to terraform.tfstate.old to check that they are the same. If not, fail the pipeline
  5. destroy the infrastructure

My question is about step 4: Compare the new terraform.tfstate to terraform.tfstate.old to check that they are the same. Obviously it won’t be sufficient to check they are the same by simply comparing file hashes because:

  1. There are various parts that are guaranteed to be different (e.g. lineage, some of the resources might have attributes that are legitimately different)
  2. The resources might not exist in the same order that they did before
  3. The new tfstate file won’t contain the outputs

So, I’m looking for an algorithm to verify that the infrastructure represented by two tfstate files are identical. Before I handcrank something like this myself, does anyone know if this is a solved problem anywhere?

2

The state file itself is an internal representation used by Terraform and therefore has no guarantees around its format. Instead there is the JSON output created by terraform show -json that is documented and designed to be used by external tools.

3

Thx @stuart-c , That should be very useful. Can probably grab the before and after JSON and compare using jq or something.

Again, if anyone has already got a working solution to this I’d appreciate seeing it.

4

Instead of comparing the state-files wouldn’t terraform plan or terraform apply reveal differences?

5

I’ve just discovered terraform state list (Command: state list - Terraform by HashiCorp) which is perfect for what I need here.

6

@jamiekt Is it possible to analyse the incremental change with that? Could you please give some more details about how you setup with some step guidance. We are also looking for the same solutions. where we are planning for below scenario

  • Our azure resources already got created using terraform with remote-state file.
  • But some manual operations and CICD pipelines changed the Actual resources states in comparison with the state file.
  • Now we used Azure Terrafy to import the actual state of the resources, so that it will create terraform manifest and new statefile.
  • So we are looking for some ways to compare the old and new statefiles (as well as the generated terraform manifests), or merge the statefiles together.

Our Aim is to reuse the existing terraform code and sync the state files inclemently for further usage.