Hey Chris !
Thanks a lot for the helpful reply. I clearly missed the obvious fact that I can use the value of the tfe_variable resource, which definitely solves my problem as I can now apply my changes in a single run.
The alternative patterns that you suggested are things I have also considered but I believe don’t fit what I am trying to achieve. I have intentionally simplified our use case to expose the problem. Here’s how our setup actually looks like:
Our infrastructure
Amazon Web Services
We are provisioning our infrastructure (essentially lambda / api gateway / eventbridge) to two AWS accounts that represent our DEV and PRO environments. While the infrastructure is very similar in both accounts, it differs on some configuration that is environment-dependent.
As an example, we are provisioning a lambda function with an environment variable that is named RESOURCE_URL whose value depends on the environment.
Terraform-Cloud repository
Our bootstrap/meta workspace
As we are currently trying to migrate from local terraform with shared backend in AWS S3 to using Terraform Cloud instead, we have created a new repository that allows us to manage our workspaces in Terraform Cloud. Here’s how this looks like for now:
// main.tf
provider "tfe" {
token = var.tfe_token
version = "~> 0.15.0"
}
// state.tf
terraform {
required_version = ">= 0.13.5"
backend "remote" {
organization = "MyOrg"
workspaces {
name = "terraform-cloud"
}
}
}
// variables.tf
variable tfe_token {
type = string
description = "The Terraform Cloud token used with the tfe provider"
}
variable oauth_token_id {
type = string
description = "The OAuth Token ID for GitHub"
}
variable organization_id {
type = string
description = "The organization id in Terraform Cloud"
}
variable aws_access_key_id_dev {
type = string
description = "AWS_ACCESS_KEY_ID for DEV"
}
variable aws_secret_access_key_dev {
type = string
description = "AWS_SECRET_ACCESS_KEY for DEV"
}
variable aws_access_key_id_prod {
type = string
description = "AWS_ACCESS_KEY_ID for PROD"
}
variable aws_secret_access_key_prod {
type = string
description = "AWS_SECRET_ACCESS_KEY for PROD"
}
variable "repositories" {
description = "List of GitHub repositories configured with Terraform Cloud"
default = [
"my-first-repository",
"my-second-repository",
"my-third-repository",
]
}
// cloud-dev.tf
resource "tfe_workspace" "tfe_workspaces_dev" {
for_each = toset(var.repositories)
name = "${each.key}-dev"
organization = var.organization_id
terraform_version = "0.13.5"
}
resource "tfe_variable" "workspace_id_dev" {
for_each = toset(var.repositories)
key = "workspace_id"
value = tfe_workspace.tfe_workspaces_dev[each.key].id
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_dev[each.key].id
description = "Workspace id"
}
resource "tfe_variable" "environment_dev" {
for_each = toset(var.repositories)
key = "environment"
value = "dev"
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_dev[each.key].id
description = "Workspace environment"
}
resource "tfe_variable" "tfe_token_dev" {
for_each = toset(var.repositories)
key = "tfe_token"
value = var.tfe_token
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_dev[each.key].id
description = "The OAuth Token ID for GitHub"
sensitive = true
}
resource "tfe_variable" "aws_access_key_id_dev" {
for_each = toset(var.repositories)
key = "AWS_ACCESS_KEY_ID"
value = var.aws_access_key_id_dev
category = "env"
workspace_id = tfe_workspace.tfe_workspaces_dev[each.key].id
description = "AWS access key id"
sensitive = true
}
resource "tfe_variable" "aws_secret_access_key_dev" {
for_each = toset(var.repositories)
key = "AWS_SECRET_ACCESS_KEY"
value = var.aws_secret_access_key_dev
category = "env"
workspace_id = tfe_workspace.tfe_workspaces_dev[each.key].id
description = "AWS secret access id"
sensitive = true
}
// cloud-prod.tf
resource "tfe_workspace" "tfe_workspaces_prod" {
for_each = toset(var.repositories)
name = "${each.key}-prod"
organization = var.organization_id
terraform_version = "0.13.5"
vcs_repo {
identifier = "MyOrg/${each.key}"
branch = "master"
oauth_token_id = var.oauth_token_id
}
}
resource "tfe_variable" "workspace_id_prod" {
for_each = toset(var.repositories)
key = "workspace_id"
value = tfe_workspace.tfe_workspaces_prod[each.key].id
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_prod[each.key].id
description = "Workspace id"
}
resource "tfe_variable" "environment_prod" {
for_each = toset(var.repositories)
key = "environment"
value = "prod"
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_prod[each.key].id
description = "Workspace environment"
}
resource "tfe_variable" "tfe_token_prod" {
for_each = toset(var.repositories)
key = "tfe_token"
value = var.tfe_token
category = "terraform"
workspace_id = tfe_workspace.tfe_workspaces_prod[each.key].id
description = "The OAuth Token ID for GitHub"
sensitive = true
}
resource "tfe_variable" "aws_access_key_id_prod" {
for_each = toset(var.repositories)
key = "AWS_ACCESS_KEY_ID"
value = var.aws_access_key_id_prod
category = "env"
workspace_id = tfe_workspace.tfe_workspaces_prod[each.key].id
description = "AWS access key id"
sensitive = true
}
resource "tfe_variable" "aws_secret_access_key_prod" {
for_each = toset(var.repositories)
key = "AWS_SECRET_ACCESS_KEY"
value = var.aws_secret_access_key_prod
category = "env"
workspace_id = tfe_workspace.tfe_workspaces_prod[each.key].id
description = "AWS secret access id"
sensitive = true
}
As you may have noticed above, we are using CLI-driven Terraform workflow for our DEV workspaces and VCS-driven Terraform workflow for our PROD workspaces.
Workspace-specific repository
Supports AWS resources for a given business domain
While we are provisioning our workspaces to the Terraform Cloud from the meta workspace as shown above, we only want to configure variables that are used across workspaces from there, while we want to configure workspace-specific variables from workspace-specific repositories.
Cross-Workspaces variables:
- AWS Credentials
- Terraform Cloud token
- Environment name
Workspace-specific variables:
Here is how things now look like in a Workspace-specific repository:
// main.tf
provider "aws" {
version = "~> 3.0"
region = "eu-central-1"
}
provider "tfe" {
token = var.tfe_token
version = "~> 0.15.0"
}
// state.tf
terraform {
required_version = ">= 0.13.5"
backend "remote" {
hostname = "app.terraform.io"
organization = "MyOrg"
workspaces {
prefix = "my-first-repository-"
}
}
}
// env.tf
locals {
environment = tomap({
dev = {
log_level = "debug"
resource_url = "my-dev-domain.com/api/resource"
}
prod = {
log_level = "error"
resource_url = "my-prod-domain.com/api/resource"
}
})
log_level = local.environment[var.environment].log_level
resource_url = local.environment[var.environment].resource_url
}
// variables.tf
variable workspace_id {
type = string
description = "Workspace id"
}
variable environment {
type = string
description = "Workspace environment"
}
variable tfe_token {
type = string
description = "The Terraform Cloud token to be used"
}
resource "tfe_variable" "log_level" {
key = "log_level"
value = local.log_level
category = "terraform"
workspace_id = local.workspace_id
description = "Log level"
}
resource "tfe_variable" "resource_url" {
key = "resource_url"
value = local.resource_url
category = "terraform"
workspace_id = var.workspace_id
description = "Resource Url"
}
// my-lambda-function.tf
resource "aws_lambda_function" "my-lambda-function" {
function_name = "my-lambda-function"
environment {
variables = {
LOG_LEVEL = tfe_variable.log_level.value
RESOURCE_URL = tfe_variable.resource_url.value
}
}
}
This is a snapshot of our current progress in the migration to Terraform Cloud. There are some obvious optimizations to be done and a few pain points, but this seem to be working and covering our requirements.
To be clear and answering your comments, I also realize that having Terraform variables for log_level and resource_url is unnecessary, but we like the idea of being able to see the current value of these variables in the Terraform Cloud UI, especially when more than one engineer may be working on the same DEV environment in the same repository.
We’d be highly interested in having your feedback on that and we’re open to share our experiences. Cheers 
