How to manage Terraform Cloud with a single terraform code and multiple VMs and Environments?

Hi,

I’ve been using Terraform for quite some time now and I have setup a specific set of Terraform code and enables me to deploy Virtual Machines to Azure. This code is used to deploy multiple servers within a single application to different environments like Dev, QA, Stage and Prod. So far, nothing special. I’ve been using Terraform CLI to deploy my infrastructure and everything is working exactly as I want and expect. However, when I try to setup Terraform Cloud Workspaces, I am running into problem. It feels like Terraform Cloud is not exactly adaptable to how I run the code.

Here’s what’s tripping me up. The TF code is a bunch of custom modules I designed to deploy my VMs that fits my needs. To deploy a set of servers for a specific application I create an app_name.tfvars file that contains variables that define a server or multiple servers. This file gets passed to the module and the module deploys the servers in a specific environment. I nicknamed that file “inventory file” and it is passed to the code using -var-file=“…/Inventory/app_name.tfvars”. I have over 50 different inventory files. Along with the inventory file, I also pass along -var=environment=DEV or -var=environment=QA or -var=environment=PROD, depending where I want the server(s) to be deployed. And one last item to make it all work is the remote state file name is also passed along to the INIT that is specific to that application and environment. So in the same physical directory, I run different INIT to point to different state files. Don’t know if that’s good or bad. Anyway, so a single application with 4 environments would have 4 different state files. Here’s an example of the CLI commands I run.

C:\Repos\Terraform\VMs\Servers
terraform init -reconfigure -backend-config=key=“app_name1.dev.isa.vm.us.tfstate”
terraform plan -var-file=“…/Inventory/app_name1.tfvars” -var=environment=DEV

or

terraform init -reconfigure -backend-config=key=“app_name1.prod.isa.vm.us.tfstate”
terraform plan -var-file=“…/Inventory/app_name1.tfvars” -var=environment=PROD

or

terraform init -reconfigure -backend-config=key=“app_name2.dev.isa.vm.emea.tfstate”
terraform plan -var-file=“…/Inventory/app_name2.tfvars” -var=environment=DEV

My Repo only has two folders, an Inventory folder where all my .tfvars files live and the folder that contains all .tf files that make it work.

When I try to configure Terraform Cloud, I just can’t see how I would adapt it to my code.

I hope this makes sense.

Hi @alexgoldberg1961,

Let me start off by saying that I’m not a Terraform Cloud experienced user myself. That being said, I do think you can adapt your setup to work with it, based on my experience with Terraform OSS.

Firstly, for each state file that you have on your local disk, you would create a new Terraform Cloud workspace under a single organization and project. You’d then migrate the local state to the cloud by adding a similar block to any .tf file and then running terraform login followed by terraform init:

terraform {
  cloud {
    organization = "my-org"

    workspaces {
      name = "DEV"   # or QA, or PROD
    }
  }
}

After that, you can add the variables in your “inventories” to Variable Sets per application and choose Apply to all workspaces in this organization.

You can also load variables from files and overwrite the variable set with per-workspace values.

If you need to share state information between workspaces, you can use the remote_state data source (though that’s discouraged when you can use the provider’s own data sources for the same purpose).

I find the Migrate State to Terraform Cloud | Terraform | HashiCorp Developer tutorial can be quite helpful in your case.