Local TFstate does not migrate to remote location (Azure Blob)

Shurbeski · December 20, 2021, 3:20pm

I am using Azure Blob to store my state. I follow these [steps] (terraform-cdk/remote-backend.md at main · hashicorp/terraform-cdk · GitHub) the only difference is that I am using the AzurermBackend. The problem is when I do terraform ini it does not migrate the existing state to the blob, it just create a new one in which there is no resources, so when i execute cdktf diff terraform says that it needs to create each resource that was already created in the local state. I checked the file the file is empty. I also tried with thr stack.addOveride that don’t works too. Next thing I did is I execute the TF_LOG=DEBUG terraform init and got the following logs:

2021-12-20T16:00:03.228+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log769761292
2021-12-20T16:00:03.228+0100 [INFO]  Terraform version: 1.0.9
2021-12-20T16:00:03.228+0100 [INFO]  Go runtime version: go1.16.4
2021-12-20T16:00:03.228+0100 [INFO]  CLI args: []string{"/usr/bin/terraform", "init"}
2021-12-20T16:00:03.228+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:00:03.228+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:00:03.228+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:00:03.228+0100 [INFO]  CLI command args: []string{"init"}

Initializing the backend...
2021-12-20T16:00:03.229+0100 [DEBUG] New state was assigned lineage "2abdb28d-45b7-02a5-d5b1-851b3c446ef3"
2021-12-20T16:00:03.229+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:00:03.233+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:00:03.233+0100 [INFO]  Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:00:03.233+0100 [DEBUG] New state was assigned lineage "ea01857e-a1b7-080a-dda5-a5081c10f48b"

Actually it just creates a new state, so I tried TF_LOG=DEBUG terraform init -migrate-state and got the following logs:

2021-12-20T16:08:07.541+0100 [DEBUG] Adding temp file log sink: /tmp/terraform-log411077971
2021-12-20T16:08:07.541+0100 [INFO]  Terraform version: 1.0.9
2021-12-20T16:08:07.541+0100 [INFO]  Go runtime version: go1.16.4
2021-12-20T16:08:07.541+0100 [INFO]  CLI args: []string{"/usr/bin/terraform", "init", "-migrate-state"}
2021-12-20T16:08:07.541+0100 [DEBUG] Attempting to open CLI config file: /home/shurbeski/.terraformrc
2021-12-20T16:08:07.541+0100 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.terraform.d/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /home/shurbeski/.local/share/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/ubuntu/terraform/plugins
2021-12-20T16:08:07.541+0100 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2021-12-20T16:08:07.542+0100 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2021-12-20T16:08:07.542+0100 [INFO]  CLI command args: []string{"init", "-migrate-state"}

Initializing the backend...
2021-12-20T16:08:07.543+0100 [DEBUG] New state was assigned lineage "4af0afde-830e-1836-4bb8-4013609be0ad"
2021-12-20T16:08:07.970+0100 [DEBUG] checking for provisioner in "."
2021-12-20T16:08:07.974+0100 [DEBUG] checking for provisioner in "/usr/bin"
2021-12-20T16:08:07.974+0100 [INFO]  Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2021-12-20T16:08:07.975+0100 [DEBUG] New state was assigned lineage "472594f8-73dc-abe6-3691-5c7bddfb715e"

Even this didn’t work.

The only thing that works if when I manually copy the tf state file and put it in the blob for the state, but i I do not like that.

Any ideas how would I get terraform to ask me if I want to migrate my pre-existing tfstate?

This is my code in the cdktf stack:

// new AzurermBackend(mystack,  {
//   storageAccountName: "cdkremotebackendtest",
//   containerName: "test1",

//   subscriptionId: "",
//   key: "terraform.tfcdk-demo.tfstate",

//   accessKey: "",


// });

jsteinich · December 21, 2021, 2:33am

Which directory are you running the terraform init command from?
The local state file is stored in cdktf.out/stacks/[stackname]/terraform.[stackname].tfstate.
Running from the stack’s output directory may allow Terraform to detect correctly, but you may also need to use the -state argument to point at the exact file.

Shurbeski · December 21, 2021, 9:46am

cdktf.out/stacks/[stackname]/terraform.[stackname].tfstate - I run it in this directory terafform init where the state file is.

terraform init -state does not work, only works with terraform plan which is not what I need.

I need terraform init -migrate-state or just terraform init to ask me if I want migrate the already existing state.

ansgarm · December 21, 2021, 1:20pm

Hi @Shurbeski!

If terraform init does not support specifying the state file via -state you could try to copy the state file terraform.[stackname].tfstate from the root directory of your CDK project into a cdktf.out/stacks/[stackname]/terraform.tfstate file and then try running the terraform init -migrate-state command in the directory cdktf.out/stacks/[stackname].

Shurbeski · December 21, 2021, 1:35pm

Hi @ansgarm

I tried that too, and the logs when I do terraform init -migrate-state is in the topic (my first post).
It starts here:

Actually it just creates a new state, so I tried TF_LOG=DEBUG terraform init -migrate-state and got the following logs:

I solved it by manually coping the tfstate from local and paste it in the the blob, which is not recommended way. That is how I “migrate it”

ansgarm · December 21, 2021, 1:40pm

Ah, got it. That’s definitely something we should improve

Topic		Replies	Views
TFC remote backend still creates local resources Terraform	3	395	February 26, 2020
How to user imported state file with Azure DevOps 'Terraform init' step Terraform	5	2315	March 4, 2021
Using terraform in an DevOps Pipeline Terraform	1	640	January 15, 2021
Azure - state file stored on storage container is not working Terraform	2	701	February 25, 2020
Help: can't initialize a remote backend with custom state Terraform	1	185	November 17, 2021

Local TFstate does not migrate to remote location (Azure Blob)

Related topics