How to manage multiple instances of the same setup?

Terraform
1

A common workflow when developing applications is to pass all changes through a sequence of environments. A common pattern might be:

  1. Development
  2. Test
  3. QA
  4. Production

Now an important feature of this is that the code deployed to production is the same code as was originally deployed to Development. We don’t copy-paste between environments we take the same exact same git commit (or build) from one environment and drop it onto the next environment.

With that in mind, we want to have our Terraform scripts follow the same workflow. As we make infrastructure changes we want to make the changes in Development; then we want to use the same script to make changes in Test, QA and finally Production. We don’t want to copy-paste between environments we want to pick up the same git commit and deploy it elsewhere.

Should be simple?

This looks trivial. Just create the scripts with variables and have a .tfvars per environment. Only there’s a catch. The backend cannot be configured via variables.. And multiple environments cannot share the same backend configuration or they will overwrite each other.

This leads us to have a directory of .tf files per environment… and that leads us to be continually copy-pasting high level changes between environments which is exactly what we don’t want.

Is there a recommended way to mange this?

2

Hey,

Have you looked into workspaces? The feature allows for multiple state files to to exist within the same working directory: State: Workspaces - Terraform by HashiCorp - but not recommended as suitable for managing multiple environments.

This is where Terraform Cloud workspaces are very well suited to environment seperation of envs, as they store individual state and you can use a different .tfvars per environment:

image
image880×960 70.7 KB

As you say, the backend cannot be configured with variables, but there is a feature called "pre-fixes’:

so in this example, the backend point to multiple terraform cloud workspaces.

terraform {
  backend "remote" {
    hostname = "app.terraform.io"
    organization = "company"

    workspaces {
      prefix = "my-app-"
    }
  }
}

On the workspace named “my-app-dev” , you would simply set an env var on that workspace named TF_WORKSPACE: “dev” and then have a corresponding dev workspace.

Hope that helps!

Tom.

3

Well as you’ve pointed out, workspaces aren’t really appropriate for multiple environments. There’s too much risk of someone deploying the wrong thing against the wrong environment’s state.

And unfortunately terraform cloud isn’t an option for us because we need some providers to access internal resources which are not exposed to the web.

If terraform can’t do this well, it feels like a pretty big miss. :worried:

4

You could look at local runners for Terraform Cloud?

5

Hi @couling,

The typical answer for a situation like yours is for each environment to have its own configuration that contains only a backend configuration, any necessary provider configurations, and a call to a shared module, like this:

terraform {
  backend "example" {
    # per-environment workspace settings
    # go in here
  }
}

provider "aws" {
  # If you're using AWS, your per-environment
  # AWS settings would belong in here.
  # Similar for any other provider that needs
  # regions/endpoint locations configured.
}

module "env" {
  source = "../modules/env"

  # per-environment configuration variables
  # go in here.
}

In this setup, the small per-environment configuration directory takes the place of the .tfvars file per environment that you considered. The variable values that you would’ve set in the .tfvars file will go inside the module "env" block instead, and the backend configuration goes inside the backend block.

The common elements that you want to have across all environments then go inside this ../modules/env directory in my example. Because they all share the same module, you can change that module and have each environment pick it up automatically next time you initialize.

1 Like