Creating AWS resources with Terraform across multiple regions

Hi,
I am using a CI/CD pipeline to automatically create multiple AWS VPC’s and resources across 2 regions (eu-west-1 & eu-south-1). The pipeline is working now but it is running in 2 parts… 1st I run the pipeline to build the VPC in the eu-west-1 using an eu-west-1 specific .tfvars file, then once that succeeds I run the pipleine a 2nd time to build the VPC in eu-south-1 using a eu-south-1 specific .tfvars file. This is obviously not the optimum solution so I am asking…
How can I merge both tfvars into one and then only need to run 1 pipeline in order to build the VPC in both regions? I have been trying to get this to work by using the for_each meta-argument but this has been unsuccessful do far. Below are the network details I am using.

eu-west-1.tfvars
vpc_cidr = “10.10.10.0/24”
subnet1_cidr = “10.10.10.0/25”
subnet2_cidr = “10.10.10.128/25”
aws_region = “eu-west-1”
my_ami = “ami-038d7b856fe7557b3”

eu-south-1.tfvars
vpc_cidr = “10.10.20.0/24”
subnet1_cidr = “10.10.20.0/25”
subnet2_cidr = “10.10.20.128/25”
aws_region = “eu-south-1”
my_ami = “ami-063c648dab7687f2b”

any help greatly appreciated.

1 Like

Hi @dubgregd,

I’d agree that root module input variables doesn’t seem like the best approach here. Root module input variables are typically for values that need to vary from one run to the next, used in situations like provider configurations, rather than fixed values that describe the infrastructure you want to create.

Instead, I’d suggest reframing your current root module as a shared module you can call multiple times. The main requirement for that would be to delete the provider "aws" blocks and any backend configuration you have in there, because those should only be declared in the root module.

Then you can write a new root module that calls your shared module twice with different settings:

terraform {
  required_providers = {
    aws = {
      source = "hashicorp/aws"
    }
  }

  # (if you had a backend configuration in your
  # old root module then move that into here too)
}

provider "aws" {
  alias = "eu-west-1"

  region = "eu-west-1"
}

provider "aws" {
  alias = "eu-south-1"

  region = "eu-south-1"
}

module "eu-west-1" {
  source = "./modules/region"

  vpc_cidr     = “10.10.10.0/24”
  subnet1_cidr = “10.10.10.0/25”
  subnet2_cidr = “10.10.10.128/25”
  my_ami       = “ami-038d7b856fe7557b3”

  providers = {
    aws = aws.eu-west-1
  }
}

module "eu-south-1" {
  source = "./modules/region"

  vpc_cidr     = “10.10.20.0/24”
  subnet1_cidr = “10.10.20.0/25”
  subnet2_cidr = “10.10.20.128/25”
  my_ami       = “ami-063c648dab7687f2b”

  providers = {
    aws = aws.eu-south-1
  }
}

In the above example I used the providers meta-argument so that each of the calls to this ./modules/region module will see a different AWS provider configuration as its default configuration, and thus any AWS provider resources you declare in there will be automatically associated with the appropriate configuration for that region.

I will note that a somewhat-typical strategy for decomposing infrastructure into multiple Terraform configurations is to split by failure domain, and regions in AWS are the top-level failure domain, so there might still be other good reasons to have a separate root module per region, though a nice thing about the above structure is that this ./modules/region module doesn’t need to know anything about how it was called and so you wouldn’t need to modify that module in order to use it from another configuration aimed at a different set of regions later, if you decided to e.g. have one configuration for Europe and another configuration for the USA: you’d just call that shared module from both configurations in that case, with different AWS provider configurations in each case.

Note that if you already have your old configuration deployed as “production” then the refactoring I described above would not be compatible with your existing state. In that case, I think the easiest path forward would be to start with a fresh new state and use terraform import to import your existing remote objects into the new resource addresses, and then discard the old separate states once you are done to preserve the assumption that each remote object is only bound to one Terraform resource instance (across all configurations) at a time.

@apparentlymart - How do we pass multiple aws provider regions/aliases into the same sub-module? For example, an RDS module that manageds replicas in multiple regions.

Hello guys, i have the same problem, follow the terraform documentation ,we can create more than one resource at once , into different region.

I create this stackoverflow post if someone can help me too:

Regards

With all due respect could someone confirm that it is not possible to create a Terraform module where the region of an S3 Bucket is determined programmatically? I’m much more familiar with the Azure provider where it is pretty straightforward to control the region of a resource, but in case of AWS - after much research - it seems to be impossible. I’m not sure that this is an issue, since it can be worked around, maybe I’m just not used to the AWS provider yet. However a clear answer from an expert would be welcome.

Thanks for your effort and hail to the developers!

@apparentlymart
While splitting across failure domains makes sense for configurations that can run separately, in the case of replication it would be a better idea to keep the logical construct in one module, example setting up s3 buckets with replication.

@acutchin-bitpusher
You can pass two providers into a submodule by using the configuration_aliases parameter in your submodules required providers, then passing in the alias when calling the module.

sub_module required providers:

terraform {
    required_providers {
        aws = {
            source  = "hashicorp/aws"
            version = "~= 4.0"
            configuration_aliases = [ aws.region2 ]
        }
    }
}

root module:

provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  alias  = "use1"
  region = "us-east-1"
}

module "sub_module" {
   ...
   
   providers = {
       aws = aws
       aws.region2 = aws.use1
   }
}

@gbuchholcz
you can determine the region programmatically, but not in the sub-module without a lot of kluges.

example of determining region programmatically:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

variable "primary_region" {
  type = string
  default = "us-east-1"
}

provider "aws" {
  region = var.primary_region
}

provider "aws" {
  region = local.replication_region
  alias = "replication"
}

locals {
  replication_region = var.primary_region != "us-west-2" ? "us-west-2" : "us-east-1"
}

data "aws_region" "primary" {}

resource "aws_s3_bucket" "primary" {
   bucket = "bucket-${data.aws_region.primary.name}"
}

data "aws_region" "replica" {
  provider = aws.replication
}

resource "aws_s3_bucket" "replica" {
  bucket = "bucket-${data.aws_region.replica.name}"
  provider = aws.replication
}
1 Like