Terraform multi AWS account deployment

Hi everyone,

I have 3 different AWS accounts; dev, qa and prod. I have main.tf to create aws vpc. Here is the code of main.tf

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "${var.environment}-vpc"
  cidr_block = "10.0.0.0/16"

  tags = {
    Terraform   = "true"
    Environment = var.environment
  }
}

I have backend.tf

terraform {
  backend "s3" {
    bucket = "${var.environment}/terraform.tfstate-test"
    key    = "terraforn.tfstate"
  }
}

I have provider.tf with profile = var.environment

provider "aws" {
  region = "us-east-1"
  profile = var.environment
}

I have AWS credentials configures in ~/.aws/credentials

[dev]
aws_access_key_id = xxxxxxxxxxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[qa]
aws_access_key_id = xxxxxxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxx
[prod]
aws_access_key_id = xxxxxxxxxxxxx
aws_secret_access_key = xxxxxxxxxxxxxxxxxxxxx

Now I want to achieve this: When I run terraform command, it should ask the value of enviroment. When I put dev, it should be applied in dev AWS account by taking dev AWS credentials and tfstate should be stored in the dev account and if I put environment value as qa, it should take qa AWS credentials and tfstate should be stored in qa AWS account and similarly for prod. I just want to run normal terraform command i.e. plan and apply

That should be fine, other than for the state file. For that you can use backend override files via the -backend-config option to terraform init.

I have 2 backends. backend-dev.tf and backend-uat.tf. It says this error

Initializing the backend...
Initializing modules...
╷
│ Error: Duplicate backend configuration
│
│   on backend-uat.tf line 2, in terraform:
│    2:   backend "s3" {
│
│ A module may have only one backend configuration. The backend was previously configured at backend-dev.tf:2,3-15.

Any given root module can only ever have a single provider, and any given directory can only have one root module, so you can’t have backend-dev.tf and backend-uat.tf in the same directory.

You probably want to be using modules: a root module per environment, then have each of those root files call out to the elements you need/want in that enviornment.

You also can’t use var.xxx in backend blocks, since backend blocks are evaluated before variables are. (You can set some backend values via command-line flags, but not within HCL itself.)

Maybe something like this? Three top-level, one named environment, one named layout, and the third components:

  • environment would be one of dev, qa, etc; there’d be at least one file in there, which I’ve called root.tf below. This provides only the things that change between environments: environment name, where state is stored, profile, etc. (Using root.tf also means that you can check for drift by simply finding all root.tf files and running terraform plan in those directories.)
  • layout, which describes what any environment should look like. This can be a single layout, or it could be a split into prod-like or dev-like or whatever.
  • component, the building blocks (which can call other building blocks!)

So something like this:

environment
  dev
    root.tf
  qa
    root.tf
  uat
    root.tf
  prod
    root.tf
layout
  prod-like
    main.tf
  dev-like
    main.tf
component
  vpc
    main.tf
  k8s
    main.tf
  ...

Then you would have a single backend spec per environment/*/root.tf, so environment/dev/root.tf might have:

terraform {
  backend "s3" {
    bucket  = "my-company-terraform-bucket"
    key     = "dev.tfstate"
    profile = "dev" # Or whatever profile is allowed to read/write to shared TF state
  }
}

provider "aws" {
  profile = "dev"
}

module "layout" {
  source = "../../layout/prod-like"

  environment = "dev"
}

Then we can specify what a prod-like environment might look like, in layout/prod-like/main.tf:

variable "environment" {
  description = "Name of environment"
  type        = string
  nullable    = false
}

module "vpc" {
  source = "../../component/vpc"

  environment = "dev"
}

module "k8s" {
  source = "../../component/k8s"

  environment = "dev"
  vpc_id      = module.vpc.id
}

Now component/vpc/main.tf could look like so:

variable "environment" {
  description = "Name of environment"
  type        = string
  nullable    = false
}

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name       = "${var.environment}-vpc"
  cidr_block = "10.0.0.0/16"

  tags = {
    Terraform   = "true"
    Environment = var.environment
  }
}

output "id" {
  description = "VPC Id."
  value       = module.vpc.id
}

(Yes, this doesn’t do much beyond what the underlying module is, and that’s generally bad form in TF – but what does do is provide useful defaults, which is one way to impose opinion / standards on your infrastructure.)

There are other approaches and other tools that aim to solve this (e.g., terragrunt). I’m not entirely sure, but I believe that Terraform Cloud “workspaces” are also trying to solve this (or a very similar) problem – but I’m not sure!

I actually use four levels: account → environment → department → service → resources.

If you know you really want to make multiple accounts look truly identical, you can sometimes play tricks “behind the back” of the provider. In my case, I wanted to enable terraform plan for non-admin users, while requiring admin privileges for terraform apply. I use a common AWS_PROFILE name, but I swap out the config file for the two use cases!

#!/bin/zsh

script_dir="${0:a:h}"

verb="$1"

if [[ "$verb" == "apply" ]]
then
  aws_mode="modify"
else
  aws_mode="readonly"
fi
echo   AWS_CONFIG_FILE="$script_dir/aws_config-$aws_mode.conf"
export AWS_CONFIG_FILE="$script_dir/aws_config-$aws_mode.conf"

echo terraform "$@"
exec terraform "$@"

This way, the backend and provider block stay constant, but the actual role/user changes. aws_config-plan.conf is:

[profile terraform]
sso_role_name = Terraform_ReadOnly
...

While aws_config-apply.conf is:

[profile terraform]
sso_role_name = Terraform_Modify
...

This means that anyone granted that Terraform_ReadOnly role can plan on the file straight as it is in source control; once they’re happy with the results, they can commit it and someone / a robot with Terraform_Modify privileges can apply it.

p.s. You’ll probably also want to look at provider-default tags, vs passing things down and adding tags to every resource.