Upgrade from v0.12.15 to v1.2 with Azure backend auth

I am trying to upgrade Terraform v0.12.15 to v1.2. I managed to upgrade to v0.13 but now I need to perform terraform init which fails with the error Error: Error building ARM Config: Error populating Client ID from the Azure CLI: No Authorization Tokens were found - please ensure the Azure CLI is installed and then log-in with az log* *in.
I understand that this is due to MS switching from ADAL to MSAL, which is the reason I wanted to upgrade to v 1.2 in the first place. So at the moment I cannot upgrade because I need a newer version but I cannot have the newer version because I have to follow the upgrade sequence.
How can I resolve this?

Hi @aegirsson,

If the Azure API is no longer compatible with Terraform v0.13 (which I’m not sure about, but your
I am aware of the old auth system being deprecated) then you may need to temporarily disable remote state and work locally to complete your upgrade.

It sounds like Terraform v0.12 is still working for you though, and so it may be worth trying to understand why that is so; Terraform v0.12’s Azure backend also predates the new “Microsoft Graph” so I’m not sure why that would work and v0.13 would not. Can you confirm that Terraform v0.12 is working for you? If so, it could at least make it easier to disable remote state to begin the upgrade; if not then it will require a more manual migration process.

Hi @apparentlymart ,

The reason I started upgrading was because it had stopped working in v0.12 so a rollback is not an option.
I ultimately just installed terraform version 1.2 with azurerm 3.21.1. Then I deleted the .terraform directories and ran the following commands:
terraform state replace-provider -auto-approve “Terraform Registry” “Terraform Registry
terraform state replace-provider -auto-approve “Terraform Registry” “hashicorp/azurerm”
terraform init --reconfigure

This seems to have forced the upgrade.
So far I have been able to reconnect to Azure and create new resources.

I do have one serious problem and that is that I had to replace the code for sql server and sql server database. Terraform changed the syntax from azurerm_sql_database to azurerm_mssql_database. And now it wants to destroy that database because it cannot find azurerm_sql_database, which is of course a major problem.

Hi @aegirsson,

Unfortunately it seems like you have hit a mixture of different compatibility problems in different components here:

  • Microsoft has removed the authentication strategy that older versions of the azurerm backend and the hashicorp/azurerm provider depend on, which forced you to upgrade both at the same time.
  • Terraform v0.13 introduced the new hierarchical provider namespace which meant you needed up update your state to use the new provider addressing scheme. However, because of the previous point you weren’t able to use the automatic upgrade tools built in to Terraform v0.13. It sounds like you were able to complete the same steps more manually using the replace-provider command though, which at least removes one problem from the list.
  • The new Azure provider version you upgraded to crossed a major release boundary for the provider and so has some breaking changes of its own. Because those belong to the provider plugin rather than Terraform itself, Terraform alone can’t automatically help with updating those.

From what you’ve described it seems like the last of these is the one remaining to fix. Unfortunately it’s also the one I’m least qualified to help with, because my focus is on Terraform Core and Terraform CLI rather than on any particular provider. :confounded:

I don’t see any mention of the particular problem you encountered in the provider’s 3.x upgrade guide, which is unfortunately what I would normally refer to in situations like this. However, one general strategy that can often work for situations like this is to use terraform import to import the existing object into a new resource of the new type, and then once you see that succeed use terraform state rm to remove the old one from Terraform’s records without destroying it in the remote system.

Unfortunately because Terraform and its providers are effectively just glue between external systems that can change at any time, it is generally impossible for us to predict how those external systems will change in future. Changes to those external systems can break behavior that older Terraform CLI or provider versions were relying on. For that reason it can be risky to wait so long to upgrade to a newer version which responds to such changes, though I’m sorry that in this case you found yourself at the center of several changes all at once which caused the documented upgrade processes for some changes to be invalidated by other changes outside of Terraform’s direct control. :confounded: Hopefully this import-and-remove strategy can get you back to a working system again. :crossed_fingers:

Hi @apparentlymart,

Yes I already had a look at the import steps.
I do think I will first try to edit the state file manually, since there are quite a few resources. It should have the same result as the import-and-remove strategy. If not, I will it with terraform import. And update terraform more often in the future to hopefully avoid these situations.

Thank you for the assistance :slight_smile:

@apparentlymart I’m having a similar issue upgrading from v0.12.31. In your second comment you talked about disabling the remote state to begin the upgrade.

Is there documentation on how to do this? Can you provide me with an example.

Hi @hgfranco,

Unfortunately I gather from the discussion above that Terraform v0.12.31 is no longer compatible with Azure because of the ADAL deprecation. Therefore I don’t think there’s any way to use Terraform itself to migrate away from remote state in the azurerm backend, because Terraform’s mechanism to migrate the latest state snapshot from one backend to another requires Terraform to be able to make requests to the old backend.

I made the suggestion to disable remote state before I understood the full cause of the problem; I don’t think that’s actually a viable solution now that I’ve learned (from the later discussion) that the ADAL deprecation has broken Terraform v0.12’s ability to interact with the Azure API altogether.

I would suggest instead trying to replicate what @aegirsson described. However, since I didn’t perform those steps myself I cannot add any more detail about how to achieve that without some more information. If you can share more detail about what error messages or other misbehavior you’re currently seeing then I may be able to make a more concrete suggestion.