Versioning modules in a monorepo

I am looking at reorganizing how my company manages our Terraform. I know the Terraform Registry recommends having one module per repo, but that seems like a pain to me, and the "Terraform Up and Running " book recommends having modules in one repo and live infrastructure in another.

How do y’all manage this? Is it a pain to manage dozens of module repos, if you do it separately? Or if you do it in a monorepo, how do you do versioning? Like if I have a module called foo-service, and it’s in my company’s Terraform repo, it seems like the URL would have to be like this:

module "foo-service" {
  source = "github.com/mycompany/terraform//foo-service?ref=foo-service-v1.0.0"
}

Is that right? It feels janky to have to repeat the module name, but I don’t know how else to do it so that the tag is specific to the module.

Or does it matter? Trying to think this through…

Hi @davidham,

The typical reason to use a monorepo is so that the entire codebase is “versioned” together as a single unit and so you can make cross-cutting changes across the entire codebase in a single commit.

For that reason, teams that use the monorepo style typically model their entire codebase as a single “Terraform package” (the term for a single container containing multiple local modules) and then use local filesystem references to call between modules:

module "foo-service" {
  source = "../modules/foo-service"

  # ...
}

It seems like you are trying for a blended model where you have a single repository but yet you pretend that it’s multiple different repositories by maintaining each module in a separate branch, with its own tags. If so, then technically you can do it but it’s an uncommon pattern and so Terraform doesn’t have any first-class support for it. The approach you showed here of specifying enough information in the source string for Terraform to know which module you meant is, I think, as concise as this approach can get.


There is one further option that is technically possible but I suspect you would consider it overkill for your problem as stated: deploy a module registry service. I’m going to describe it for completeness, because I think it’s the only way to achieve your goal of having a concise source string even though your modules are in an unusual layout.

A Terraform module registry is essentially just a collection of JSON indexes that allow a remote system to own the rules for going from a module name and a version number to a specific physical source location. If you implement Terraform’s module registry protocol then you can implement the “Download Source Code for a Specific Module Version” operation to return a Git source string like you showed in your question, allowing you to build those strings systematically rather than manually in each case.

If you deployed a registry at terraform.example.com then a call to your foo-service module for AWS might look like this:

module "foo-service" {
  source  = "example.com/mycompany/foo-service/aws"
  version = "1.0.0"
}

Terraform would then send that information to your registry, which would then know how to construct the GitHub URL based on the module name and the version number:

X-Terraform-Get: git::https://github.com/mycompany/terraform//foo-service?ref=foo-service-v1.0.0

The idea of each module belonging to a “system” like “aws” is a fundamental part of the protocol aimed at allowing you to have multiple implementations of the same module across different remote systems, but if you have no need for that right now (e.g. because you only use one cloud system) then you could have your registry just hard-code it to whichever system you currently use and ignore it when constructing URLs.

(Unfortunately at the time I’m writing this the Module Registry Protocol docs contain some bugs due to some earlier editorial errors that we’re currently working on correcting. Specifically, some of the URL schemes include both :provider and :system placeholders, where in fact :system was supposed to replace :provider as part of an effort to make it explicit that the “system” part is only a provider type by convention, and that modules don’t actually physically belong to a single provider. The docs will be updated soon, but for now if you intend to implement the protocol then please disregard those redundant extra entries in the path templates.)