I am working on an open source project with Terraform that will allow me to set up ad-hoc environments through GitHub Actions. Each ad-hoc environment will correspond to a terraform workspace. I’m setting the workspace by exporting TF_WORKSPACE before running terraform init, plan and apply. This works the first time around. For example, I’m able to create an ad-hoc environment called alpha. In my S3 backend I can see that the state file is saved under the alpha folder. The issue is that when I run the same pipeline to create another ad-hoc environment called beta, I get the following message:
Initializing the backend...
╷
│Error: Currently selected workspace "beta" does not exist
│
│
╵
Error: Process completed with exit code 1.
Here is the section of my GitHub action that is failing: django-step-by-step/ad_hoc_env_create_update.yml at main · briancaffey/django-step-by-step · GitHub
I have been over this article: Selecting a workspace when running Terraform in automation – HashiCorp Help Center but I’m still not sure what I’m doing wrong in my automation pipeline.
The alpha workspace did not exist, but it seemed to be able to create it and use it as the workspace in my first run. I’m not sure why other workspaces are not able to be created using the same pipeline.
Hi @briancaffey,
Unfortunately the part of this which is surprising to me is that this worked for your alpha workspace, since indeed Terraform refusing to work with a non-existing workspace is by design so that Terraform can catch typos that might, for example, lead to creating an entirely new parallel set of infrastructure when one intended to update some existing infrastructure.
To make this work you’ll need to introduce an extra step into your pipeline to explicitly create the workspace. Workspace creation requires having the backend initialized first, so you’ll need to wait until after terraform init and terraform workspace create before setting the environment variable. The sequence of steps would therefore be:
terraform init ...terraform workspace create "${WORKSPACE}"export TF_WORKSPACE="${WORKSPACE}"terraform apply ...terraform output ...
(Note that terraform apply -auto-approve already has the Terraform planning step built into it if you aren’t also providing a saved plan file, so the extra terraform plan step before that isn’t adding anything in your current pipeline; you’re just running the planning process twice.)
1 Like
Hi @apparentlymart, Thank you for your advice on this. I’m also puzzled about why the first workspace was able to be created, but a second one was not (using only TF_WORKSPACE). I will update my GitHub Actions script to do things in the order that you recommended and post back here with what I find.
I had tried doing something like:
terraform workspace select $WORKSPACE || terraform workspace new $WORKSPACE
and other combinations of these commands, but I’ll make sure I’ll try doing this in the order shown in your reply. This does make a lot of sense though. If the workspace doesn’t exist then I will need to create it.
I also saw some interesting solutions in this thread: terraform init doesn't honor input or automation flags for invalid workspace · Issue #21393 · hashicorp/terraform · GitHub, but it looks like it may be specific to an older version of Terraform.
Thanks for pointing that out about terraform apply. I see that I don’t need it here in the way that I’m using it.
This worked perfectly @apparentlymart, thank you again! The key part that I wasn’t doing in some of the other ways I had tried to make this work is exporting the TF_WORKSPACE AFTER trying to create the workspace. I’m now doing things in the same order that you mentioned:
terraform init ...
terraform workspace create ${WORKSPACE} || echo "Workspace ${WORKSPACE} already exists or cannot be created"
export TF_WORKSPACE=$WORKSPACE
terraform apply ...
terraform output ...
Now it has the correct behavior of “create the workspace if it doesn’t exist, or select the existing workspace” and I can create multiple ad hoc environments in my project without pipeline failures! 
@briancaffey I’m assuming your GitHub Actions are working in a freshly cloned workspace each build. You comments imply you are ably to assume that at the beginning of your pipeline there isn’t already a workspace selected (other than default).
FWIW, if anyone ends up here with this issue and is not able to make this assumption (bc your running on a dev workstation or a pipeline node that’s re-using the workspace without cleaning up .terraform after a workflow that switches commits/branches regularly, probably an anti-pattern, but…)…
…it seems to me you can’t always just terraform init ... as a first step, bc the currently selected workspace might not exist. This is an insidious situation, since it deadlocks the proposed sequence above, even when explicitly unsettting TF_WORKSPACE prior, because:
- running
terraform init first → Error: Currently selected workspace "X" does not exist
- trying to select a known-good workspace first, e.g.
terraform select default, → Backend initialization required: please run "terraform init"
I run into this from time to time and the only solution I know of is another anti-pattern which is to pull back terraform’s curtain a bit and hack away at its internal implementation details for “selecting a workspace” as a first step. (This is actually Option 2 in the page you linked earlier.)
function init_workspace {
local _TF_INIT_EXTRA_ARGS=${1:-}
echo "Selecting workspace..."
printf '%s' "default" > ${SCRIPT_DIR}/../.terraform/environment
SAVED_TF_WORKSPACE=$TF_WORKSPACE
unset TF_WORKSPACE
terraform init -input=false ${_TF_INIT_EXTRA_ARGS}
TF_WORKSPACE=$SAVED_TF_WORKSPACE
terraform workspace select "${TF_WORKSPACE}" || terraform workspace new "${TF_WORKSPACE}"
}