A smarter dev than me has used a GitLab CI variable to generate a backend.hcl file, with workspaces { name = "${WORKSPACE}" } , which works for now. But I’m curious about this.
The “remote” backend (which is what integrates Terraform CLI with Terraform Cloud/Enterprise) can operate both in multi-workspace mode and in single-workspace mode.
When you set name in the workspaces block, that activates (from Terraform CLI’s perspective) single-workspace mode, in which case Terraform CLI ignores Terraform Cloud’s idea of workspaces altogether and behaves as if there is a single workspace called default which maps directly to the remote workspace you specified. This mode is there primarily for folks who were already using Terraform Cloud’s predecessor via a different API.
If you instead set prefix in the workspaces block then that’ll activate the multi-workspace mode, which allows Terraform CLI and Terraform Cloud’s idea of workspaces to work together, but this approach also expects you to follow a particular naming convention for your workspaces in Terraform Cloud of using the same prefix for all of the workspaces that share a particular root module directory. You’d need to use this mode and follow the expected naming convention in order to be able to dynamically switch between remote workspaces using the Terraform CLI commands.
In your second comment here you described a script generating a backend configuration file, and that generated file is activating the single-workspace mode where Terraform CLI sees only a single workspace called default, which maps to whatever remote workspace name you specified there. terraform.workspace will therefore return default, and if you were to run terraform workspace list you’d see it indicate that “default” is the only workspace that exists.
If you use the multi-workspace mode instead, you’ll see that terraform workspace list will indicate one Terraform CLI workspace for each remote workspace that shares the given prefix. At that point, you can switch between them dynamically either by running terraform workspace select (which, internally, tracks the current workspace name in a file under the .terraform directory), or by setting the environment variable TF_WORKSPACE to one of the names you see in the terraform workspace list output.
With that said, your current approach of generating a file with a specific name in it is fine too, and has the advantage of not requiring that you follow the prefix-match naming scheme that the remote backend normally requires for multi-workspace mode. The downside of it is that then Terraform CLI isn’t aware of the multiple remote workspaces, though that often isn’t a huge problem in practice.