Any given root module can only ever have a single provider, and any given directory can only have one root module, so you can’t have backend-dev.tf and backend-uat.tf in the same directory.
You probably want to be using modules: a root module per environment, then have each of those root files call out to the elements you need/want in that enviornment.
You also can’t use var.xxx in backend blocks, since backend blocks are evaluated before variables are. (You can set some backend values via command-line flags, but not within HCL itself.)
Maybe something like this? Three top-level, one named environment, one named layout, and the third components:
environment would be one of dev, qa, etc; there’d be at least one file in there, which I’ve called root.tf below. This provides only the things that change between environments: environment name, where state is stored, profile, etc. (Using root.tf also means that you can check for drift by simply finding all root.tf files and running terraform plan in those directories.)layout, which describes what any environment should look like. This can be a single layout, or it could be a split into prod-like or dev-like or whatever.component, the building blocks (which can call other building blocks!)
So something like this:
environment
dev
root.tf
qa
root.tf
uat
root.tf
prod
root.tf
layout
prod-like
main.tf
dev-like
main.tf
component
vpc
main.tf
k8s
main.tf
...
Then you would have a single backend spec per environment/*/root.tf, so environment/dev/root.tf might have:
terraform {
backend "s3" {
bucket = "my-company-terraform-bucket"
key = "dev.tfstate"
profile = "dev" # Or whatever profile is allowed to read/write to shared TF state
}
}
provider "aws" {
profile = "dev"
}
module "layout" {
source = "../../layout/prod-like"
environment = "dev"
}
Then we can specify what a prod-like environment might look like, in layout/prod-like/main.tf:
variable "environment" {
description = "Name of environment"
type = string
nullable = false
}
module "vpc" {
source = "../../component/vpc"
environment = "dev"
}
module "k8s" {
source = "../../component/k8s"
environment = "dev"
vpc_id = module.vpc.id
}
Now component/vpc/main.tf could look like so:
variable "environment" {
description = "Name of environment"
type = string
nullable = false
}
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
name = "${var.environment}-vpc"
cidr_block = "10.0.0.0/16"
tags = {
Terraform = "true"
Environment = var.environment
}
}
output "id" {
description = "VPC Id."
value = module.vpc.id
}
(Yes, this doesn’t do much beyond what the underlying module is, and that’s generally bad form in TF – but what does do is provide useful defaults, which is one way to impose opinion / standards on your infrastructure.)
There are other approaches and other tools that aim to solve this (e.g., terragrunt). I’m not entirely sure, but I believe that Terraform Cloud “workspaces” are also trying to solve this (or a very similar) problem – but I’m not sure!
I actually use four levels: account → environment → department → service → resources.
If you know you really want to make multiple accounts look truly identical, you can sometimes play tricks “behind the back” of the provider. In my case, I wanted to enable terraform plan for non-admin users, while requiring admin privileges for terraform apply. I use a common AWS_PROFILE name, but I swap out the config file for the two use cases!
#!/bin/zsh
script_dir="${0:a:h}"
verb="$1"
if [[ "$verb" == "apply" ]]
then
aws_mode="modify"
else
aws_mode="readonly"
fi
echo AWS_CONFIG_FILE="$script_dir/aws_config-$aws_mode.conf"
export AWS_CONFIG_FILE="$script_dir/aws_config-$aws_mode.conf"
echo terraform "$@"
exec terraform "$@"
This way, the backend and provider block stay constant, but the actual role/user changes. aws_config-plan.conf is:
[profile terraform]
sso_role_name = Terraform_ReadOnly
...
While aws_config-apply.conf is:
[profile terraform]
sso_role_name = Terraform_Modify
...
This means that anyone granted that Terraform_ReadOnly role can plan on the file straight as it is in source control; once they’re happy with the results, they can commit it and someone / a robot with Terraform_Modify privileges can apply it.
p.s. You’ll probably also want to look at provider-default tags, vs passing things down and adding tags to every resource.