Why does a plan lock the state file?

(using Terraform v0.11.14)

A question regarding the default behaviour in terraform: why does a terraform plan default to locking the state file? As the locking document says:

If supported by your backend, Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state.

Yet the plan operation does not write state (and could not , AFAIK).

Is this an anachronism, left over from when it did write state?

2 Likes

Hey @leg100,

This is a good question! My understanding is for the reason for Terraform locking state during a plan is due to the auto-trigger of terraform refresh as part of the plan.

Terraform will review the existing state and ensure that the resources that are defined in statefile are as explicitly declared in HCL.

For example, I create a vanilla AWS EC2 instance via Terraform. Then I go to the AWS UI and manually add a tag to the instance. On the next plan after than manual change has happened. The refresh part of the plan would state detect the drift (removal of the tag) due to the manual UI change.

Hopefully, that clears things up. Otherwise, as you know - there is always the official documentation.

1 Like

Hi @mpmsimo

Thanks for the response. But I don’t think you’re quite right.

If you run a plan, you’ll immediately see the following message:

Refreshing Terraform state in-memory prior to plan…
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

I believe what you say was once correct, a plan did in effect run terraform refresh, which does persist to state. But, like I said in my original question, a plan no longer does this, instead refreshing the state ‘in-flight’.

So the question still stands…why lock?

1 Like

@leg100, looks like you suspicions are correct.

As you’ve said the terraform refresh and terraform plan are done in different phases. I had previously referenced the Terraform documentation which didn’t go into the level of detail we wanted. In this case it made sense to review the codebase.

Enforcing a locked statefile is the default behavior when using terraform CLI plan command.

As locking is intentional, explicitly passing in the -lock=false argument to the CLI command is the way to override this behavior…

I know this is the default (it is documented, there is no need to go into the code). There is no reason for it to be offered as an option, AFAICS.

It would be good if someone familiar with the history of terraform or a core contributor could answer this.

1 Like

I have had locking help prevent interruption if multiple people are accessing the same statefile.

I will leave it up to someone more knowledgeable on this topic to give you an exact answer, since I don’t have that historical context of the codebase.

Is the locking during plan not simply a way to avoid someone else’s writes from corrupting your plan if they happen during the read of the current state?

@Richard-Payne A read is an atomic operation, so no.

There are a dozen backend types, and I haven’t looked into the ACID properties of each and every one, but it’s fair to assume that any database or filesystem worth its salt will guarantee to avoid what you describe.

If what you describe were true, you’d never have confidence reading from anything!

1 Like

I was wondering the same thing. According to the documentation though, terraform refresh does update the state:

This does not modify infrastructure, but does modify the state file. If the state is changed, this may cause changes to occur during the next plan or apply.

1 Like

From Terraform plan should not lock state files by default since it no longer writes changes to state · Issue #28130 · hashicorp/terraform · GitHub

Thanks for filing the request. Just to provide a quick answer for why we can’t immediately implement this; the way the remote state backend code is structured, remote state implementations can still write to their underlying storage when requesting access to a state, even if it’s not writing the terraform state itself. For this reason, internally when locking is possible, the implementation must take the lock to update the storage structures.

Another type of consideration is that if an apply is happening concurrently to the plan operation, the plan may show partially or fully invalid results. Now if this plan is saved, it could of course not be applied since the state would not match the new changes, but the result would be confusing to users who are assuming only one operation can happen at a time.

We are however working on restructuring remote state storage in general, and specifying all the requirements and details around locking of state storage will be part of that project. Once the new specification for remote state locking is defined, it will likely guide any changes to how state is locked in the CLI.

2 Likes

Does terraform plan have a READ LOCK such that terraform plan can have a write lock which would allow MANY people do grab a read lock and do a terraform plan at the same time but no writer. And if a writer grabs the write lock, all readers are locked out? That would be very very nice.