Apply different infrastructure to different environments

Terraform
1

Hi All,

I’m trying to understand the best way to utilise Terraform to create multiple environments in AWS (I’m currently using workspaces). This works fine when the environments are the same, say for dev/test, but when I build my Prod environment, the infrastructure will change. I.E deploying to 3 AZ’s instead of 1 etc.

Similar questions to this has been asked a lot I feel but I’m just trying to figure out the best options. I obviously want to re-use the same code as much as possible and my codebase is modular currently. Is the best option to create separate root modules for test and prod and then separate child modules where my infrastructure has to differ? Or is there a better way?

2

We use variables & tfvar files (which we store in git), one per environment (and usually one for values which are common for all environments). Our CI process then picks up the correct file when running (and in our case we also use workspaces, so the tfvar file corresponds with a specific workspace).

3

How does that allow you to build different infrastructure per environment though? Have you got an example I can see?

4

The different variables would be used to configure whatever is being produced.

For a VPC you might pass in the CIDR, but could also configure the number of subnets or subnet tiers (public, private, persistence).

For RDS it might determine the instance sizes, storage and if to create a clustered version.

As a result using the exact same code you could deploy a development environment with lower spec systems and no high availability, at the same time as deploying a production environment with higher specifications.

5

I’m sure I am missing something trivial here that is not making me understand how this can be used. As an example, within my ECS module for instance I use something like this:

desired_count = (lower(terraform.workspace) == "prod") ? 2 : 1

Which is how I differentiate how many ECS tasks I want to run, I do similar things with RDS etc. But if I was to pass in a value as to your example into my vpc module to create a single subnet for dev and two subnets for prod, I’m still not sure how to achieve that.

Is there a ‘if blah then blah else blah’ syntax in terraform I’m not seeing?

6

I ended up solving my specific use case by just having count variables looking at the workspace for each piece of infrastructure where I need multiples built. I.E:

resource "aws_eip" "ApplicationNATGatewayEIP" {
  count = (lower(terraform.workspace) == "prod") ? 2 : 1
  vpc   = true
}

This seems to work well for what I need and I don’t require any extra var files or anything. It’s not as flexible as it probably could be but nonetheless it meets my needs.

7

The count (or for_each) entries are exactly what you need.

For example to selectively make a resource based on a variable it would be

resource "xxx" "yyy" {
    count = var.make_resource ? 1 : 0

    ...
}

So to make different subnets you just do something similar with the different resources needed. As an example (and also a handy set of modules to use) take a look at Terraform AWS modules · GitHub (specifically GitHub - terraform-aws-modules/terraform-aws-vpc: Terraform module which creates VPC resources on AWS)

8

Hi @tsposato,

The Terraform documentation section When to Use Multiple Workspaces discusses the limitation you encountered here, and describes the typical way to address it: write a separate root module for each of your environments, and then use shared child modules to represent what those environments have in common.

You can then represent the differences between the environments by techniques such as instantiating different combinations of modules, or by passing different arguments to some of the modules. This also tends to be a good situation to employ the Module Composition techniques.

Although in principle you can use multiple workspaces to manage multiple environments, the purpose of the workspaces mechanism is to deploy essentially an identical copy of the whole declared infrastructure, with only minor differences for namespacing purposes, and so it’s not a suitable answer for situations where the environments have non-trivial differences between them, as seems to be the case for you.