Confusing error message when terraform backend is changed

Terraform
1

I am using s3 as terraform backend to manage AWS resources, and recently I enabled SSO for AWS by adding a SSO profile in the s3 provider as below.

  backend "s3" {
    bucket = "test-bucket-xxx"

    key     = "terraform"
    region  = "us-west-2"
    profile = "xxx"
  }

Error: Backend configuration changed

A change in the backend configuration has been detected, which may require migrating existing state.

If you wish to attempt automatic migration of the state, use “terraform init -migrate-state”.
If you wish to store the current configuration with no changes to the state, use “terraform init -reconfigure”.

What does “migration” mean here? If I change to use a different a backend(e.g. a different s3 bucket name), does “terraform init -migrate-state” migrate the state from the old bucket to the new one? What if the old bucket does not even exist anymore? Or does it migrate from a local cached version? What if I changed the terraform version (.i.e. required_version) or/and the aws provider version? Will it cause the same error? I asked this because I am not sure if terraform will need to change the data format in the backend.

I am also super confused by the If you wish to store the current configuration with no changes to the state, use "terraform init -reconfigure. What does the current configuration mean? It is what I have in the .tf file or it is the AWS resources I have set up?

3 Likes
2

Hi @alexi.zuo,

I think some important background information to understand what’s happening here is that a successful terraform init “remembers” the backend config it found (the merger of the backend "s3" block and any -backend-config command line options) in the internal metadata file .terraform/terraform.tfstate, which all subsequent Terraform commands in this working directory then use as “the [backend] configuration” which this error message is discussing to.

So Terraform has found an existing .terraform/terraform.tfstate file in your working directory and noticed it contains different settings than your backend "s3" block, which is what it means by “Backend configuration changed”.

“Migrating existing state”, then, is referring to the possibility of reading the latest state snapshot using the old configuration (from the .terraform/terraform.tfstate file) and then writing it using the new configuration (your backend "s3" block, which it’s calling “the current configuration”). As you noted, that would only make sense if you’d changed the configuration in a way that means the physical artifact will be in a different place, and even then it would work only if there’s still a snapshot accessible at the old location.

In your case you seem to have changed a setting that only affects how the S3 backend will authenticate to AWS, and not which S3 bucket and object path it will be using, and so migration here (using -migrate-state) would be a no-op at best: it would just try to read the state using your old credentials and then write it back to the same location using the new credentials. In practice, I expect it would probably fail because your old credentials presumably aren’t valid anymore due to your new requirement for SSO.

That then brings us to -reconfigure: roughly speaking, that’s essentially the same as what would happen if you deleted the .terraform/terraform.tfstate file before running terraform init, causing Terraform to think it’s just initializing this working directory for the first time. (There are some minor differences in other situations relating to legacy Terraform version upgrade paths that don’t apply to you here, but what I’ve said in this paragraph is true for your situation.)

Given that, you could to one of two things here, each of which will achieve the same result of just making Terraform start using your configured profile name:

  • Run terraform init -reconfigure to tell Terraform to just use what’s in the backend "s3" block and disregard the .terraform/terraform.tfstate file altogether.
  • Delete the .terraform/terraform.tfstate file (or, if you like, the entire .terraform directory) to erase Terraform’s memory of having initialized this directory before, therefore forcing it to just use the current backend "s3" block contents with no awareness that there might’ve been any previous backend configuration.

This message is vaguer than it would ideally be just because terraform init doesn’t really know anything about the S3 backend in particular, and so it’s just working generically with the opaque configuration values and not taking into account what any of them might happen to mean as far as the S3 backend is concerned. Unfortunately that means that we as users need to think about the consequences of what changed in the configuration and decide whether copying the state from old to new will be helpful (and possible) or not, which is the decision this error message is inviting you to make.

8 Likes