Broad questions about Terraform and monorepos

Terraform
1

I’m currently managing some infrastructure that is organized like this:

  • One terraform config repository per “area”
  • Dozens of repositories for modules that are re-used in these config repositories

The modules are tagged and the config repos pin the tags. So when a module is updated with some fix and the version is bumped, say, from 3.0.0 to 3.0.1, then we need PR’s for all config repositories updating that. If the change requires the recreation of resources, then that might need to be coordinated with other teams that own the applications running in each area.

This works fine but it has some problems:

  • Many PRs for a simple change to some module (PR for module change, PR to update terraform config’s development environment, PR to update the production environment).
  • It’s easy for configuration to be left behind (some module instantiations using v3.0.0 while others using the new v3.0.1)
  • Lots of permissions/webhooks to manage for each Git repository.

So I was thinking about how a single monorepo for the whole infrastructure would look it. Anyone doing this at a large scale with Terraform specifically?

Sometimes the pinning helps because we can update certain parts of the infrastructure and not others, when it’s convenient. But it creates technical debt. For instance, a module that is updated to follow some best practice doesn’t get used across the whole infrastructure and could cause problems later on when breaking changes start to pile up.

What’s your opinion on this? Could anyone share how they are using monorepos with Terraform? And how do you handle applying changes to the whole infrastructure? Are you able to “pin” things even if live config and modules live side by side in the same repo? Multiple branches?

And finally, if you don’t use a monorepo and prefer the arrangement I described above, how do you make it less painful?

2

I’m way behind in my un-read messages :slight_smile: . We use a mono repo and have each ‘project’ in its own directory. We have a common set of our modules in a top level ‘modules’ directory. This is in itself is easy to manage. The tricky part is the release management aspect.

Each PR has its own git branch. After the PR is approved and sent through version control, a terraform plan is generated and saved against each of our environments. When we promote the change through the environments, we run terraform against the saved plan file for that environment. Also we only run/save plans for projects that have changed. We detect the changed projects based on the git changes. It isn’t perfect, for example, if you change any common modules/files, things aren’t automatically updated based on those git changes. It requires a global rebuild at that point :confused: . Hope something in here is useful for you

1 Like
3

That’s very useful, thank you!