Authentication to Terraform Cloud remote backend (from CI/CD - Azure Pipelines)

Terraform HCP Terraform
1

Hi,

We are using workspaces and the private module registry in Terraform Cloud.

The particular workspace in question, is configured to use the ‘Local’ execution mode.
(Your plans and applies occur on machines you control. Terraform Cloud is only used to store and synchronize state.)

The workspace code implements the terraform remote backend in the following way:

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "Our-Organization-Name"
    
    workspaces {
      name = "Our-Workspace-Name"
    }
  }
}

This work as it’s supposed to (authenticates to the remote backend, makes use of the state file, and downloads a module from the private module registry), on my local machine, when I use any of the following authentication methods (with my user API token) :

Now to the question/issue:

We want to make use of Terraform in our CI/CD system (Azure Pipelines).
This rules out the interactive terraform login, and also the CLI config file. This because we want to be able to pass in the token in as a parameter.

The documentation covering the remote backend, states (under the following section; Configuration variables) that it should be possible to use the token configuration variable to configure the access token. If I specify this token token variable like so:

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "Our-Organization-Name"
	token = "TOKEN.atlasv1.TOKEN"
	
    workspaces {
      name = "Our-Workspace-Name"
    }
  }
}

I get the following error:
image

I’d like to be able to use the ‘Command-line key/value pairs’ functionality, to be able to specify a partial configuration (when run in CI/CD), like so:

terraform init -backend-config="token=TOKEN.atlasv1.TOKEN"

But this, unsurprisingly, gives the same error as above:
image

Am I doing something wrong when configuring the token configuration variable?

Which is the prefered way of authenticating to the Terraform Cloud from a CI/CD system (in this case Azure Pipelines)?

Any input is appreciated!

Best regards.

2

Hi @aleqsss! It looks like you might be running into this Terraform bug, which I’ve been able to reproduce locally today.

I think the core problem is a caching bug for the initialized backend. Please try removing the .terraform/terraform.tfstate file (perhaps renaming to .terraform/terraform.tfstate.backup). This should flush the remote backend cache, and then the command you tried should work:

terraform init -backend-config="token=TOKEN.atlasv1.TOKEN"

Can you give this a try and see if it works? If not, it would help if you could either add some details to the existing Terraform issue if it seems like the same problem, or open a new issue if not.

3

Hello @alisdair,

I think that you are on to something, regarding your investigation on the terraform issue. :slightly_smiling_face:
Your proposed workaround worked for me, so you can consider this solved.

Thank you for your time and solution.