Is there a way to somehow track the history of resources defined in Terraform?
Imagine I have a big main.tf, with a lot of modules, etc. Now, I want to see how resource X has been modified over time.
Via git - this will be really really hard.
1 Like
This is def available in TF Cloud and I assume Enterprise.
You can DIY using object storage for remote state if you have versioning enabled on the bucket too.
Hi @random930,
As @kunickiaj implied, whether historical state versions are available depends on which state storage backend you are using and how you have it configured. Terraform does not retain historical state versions itself, but the storage backend might do so.
Terraform Cloud and Terraform Enterprise both have a state storage backend which retains all historical versions and has a UI and API for browsing them.
For the s3 backend you can turn on versioning on your S3 bucket to get a similar effect, but it’s S3 providing that rather than Terraform itself.
Some of the other state storage options offered by Terraform might also have versioning capabilities but, as with S3, that would be something you’d need to enable within the storage system itself rather than within Terraform.
Thank you both. The versioning of states (whether via TF Enterprise or S3 buckets), grabs the entire state. Is there an easy way to pick a specific resource and see its history? Imagine if each resource was in their own file - then it would be easy to do it via git.
Do any of the above approaches allow an easy view of the history of a specific resource?
Not an answer to your specific question, but if using AWS, I will enable AWS CloudTrail as it records all changes and API calls making possible to see all resources history.
I think I would likely do that by retrieving a pair of historical state versions and using a small program to extract only the subset of data I were interested in and then compare the results of that program.
For one-off research I expect I would do it by extracting data using a tool like jq.
Technically this sort of programmatic comparison is not guaranteed to work well forever because the state snapshot serialization format changes from time to time and would thus introduce churn that isn’t reflective of something actually changing in the remote system, but the state serialization format has only significantly changed twice in the history of Terraform (once to switch from a binary encoding to JSON very early on, and then in v0.12 to support the new type system) so it shouldn’t take a lot of maintenence to keep a tool like that working with newer Terraform versions.
(In a previous job I actually did something similar: I wrote a little program that would generate an HTML description of infrastructure based on the state so it’d be easier for humans to skim and see what infrastructure belonged to a particular subsystem. That was prior to Terraform v0.12 and so it would’ve required some non-trivial adjustments to continue working after Terraform v0.12, but otherwise I only made a couple minor adjustments to it throughout upgrades from Terraform 0.5 to 0.8, at which point I left that job and started working on Terraform at HashiCorp. 
)