I am somewhat new to Terraform and looking for suggestions on how to better structure or manage our Terraform deployments.
We will be deploying > 20 AWS Elastic Kubernetes Service (EKS) clusters across different accounts and regions, each cluster with different configuration settings.
Making use of Terraform Cloud, each cluster has it’s own git repo, state file and uses shared modules hosted in the TF Cloud registry.
The main structure for clusters looks like this.
clusters
├── eks-cluster1
│ ├── backend.tf
│ ├── data.tf
│ ├── locals.tf
│ ├── main.tf
│ ├── output.tf
│ ├── providers.tf
│ └── variables.tf
├── eks-cluster2
│ ├── backend.tf
│ ├── data.tf
│ ├── locals.tf
│ ├── main.tf
│ ├── output.tf
│ ├── providers.tf
│ └── variables.tf
└── eks-cluster2
├── backend.tf
├── data.tf
├── locals.tf
├── main.tf
├── output.tf
├── providers.tf
└── variables.tf
Each main.tf file calls up to 9 shared modules that performs a specific task, and it all works well. Here is a basic example.
module “eks-cluster” {
source = “app.terraform.io/ACME/eks-cluster/aws”
version = “2.0.1”
}
module “eks-nodegroup” {
source = “app.terraform.io/ACME/eks-nodegroup/aws”
version = “1.1.0”
depends_on = [module.eks-cluster]
}
module “eks-nginx” {
source = “app.terraform.io/ACME/eks-nginx/aws”
version = “1.2.14”
depends_on = [module.eks-nodegroup]
}
At this time I have two concerns as we continue to scale out this solution.
1. Drift in the root module.
As additional features or functionality are added, each root module will need to be manually updated and I can’t really think of a way to verify that the root modules are the same.
How would you manage drift in the root modules?
2. How do you manage module versions?
As changes are made to a module, it’s Git version is incremented and rolled out to the different clusters and environments. This would be much easier to manage if module versions could be referenced from variables, but that’s not supported. If we start with 20 clusters and each cluster has 9 or modules configured, that’s 180 version number scattered across multiple main.tf files to keep track of and update.
How are you managing module version numbering for large deployments?
As of now, I wrote a Python script to parse all the module versions in clusters\eks-clusterX.terraform\modules\modules.json and save to a csv file. That will at least give me a summary report of the modules used and their current versions.
Any suggestions would be greatly appreciated.