How to setup a mono-repo for terraform modules?

Terraform
1

We have several private modules that we want to keep private. We use terraform registry for this, but we have poly repos, over 47 of them. It would be nice to keep just one single repo for all of our modules, and then have terraform registry use these.

How do we do this?

2

Hi @darkn3rd,

The VCS integration mode for private module registry assumes that repositories will follow the standard module structure, and therefore works best with one repository per module package.

However, if the built-in behavior isn’t suitable for your situation then you can alternatively push locally-prepared module packages into the registry using the API, in which case Terraform Cloud will not interact with your VCS at all and so you can arrange your VCS however you like as long as you have some release automation scripts that can construct an archive for each module package that follows the standard structure.

There’s some more detail on this in the private module registry API docs:

If subdirectories in your monorepo still follow the standard module package structure within them then you should be able to directly archive the relevant subdirectory and submit the result verbatim to the upload endpoint. These steps are essentially what the automatic VCS integration would normally do except that it only knows how to do it from the root of the repository, including all of the content from the repository in the uploaded package.

3

We are using the automation from the tf cloud, so this means tagged poly repo.

That makes sense to use a push API, then you could use whatever scheme you want.

4

Hi @apparentlymart,

I implemented the steps that you provided which allowed me to publish all the modules from my mono-repo setup to our private registry (which was a success). I have a question mostly about running terraform validate after I reference the modules. It appears that all of the modules fail with error that all the fields are not expected in the module. Below you can see the code which after running terraform validate fails. Also would add that running terraform validate multiple times every time the commands outputs different module that contains the errors, not all the errors at once (once security_group, than ASG, and so on).

Example module code:

module "example_sg" {
  source  = "app.terraform.io/<organization_name>/security_group/aws"
  version = "1.0.0"

  environment   = var.all_vars.environment
  vpc_id        = module.vpc.vpc_id
  name          = var.example_vars.security_group.name
  description   = var.example_vars.security_group.description
  ingress_rules = var.example_vars.security_group.ingress_rules
  egress_rules  = var.example_vars.security_group.egress_rules
  tags          = merge(var.all_vars.default_tags, var.example_vars.autoscaling_group.tags, var.example_vars.security_group.tags)
}

Error output:

│ Error: Unsupported argument
│ 
| ...
│ 
│ An argument named "environment" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
| ...
│ 
│ An argument named "vpc_id" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
| ...
│ 
│ An argument named "name" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
| ...
│ 
│ An argument named "description" is not expected here.
╵
╷
│ Error: Unsupported argument
│  
| ...
│ 
│ An argument named "ingress_rules" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
| ...
│ 
│ An argument named "egress_rules" is not expected here.
╵
╷
│ Error: Unsupported argument
│ 
│   on security_groups.tf line 416, in module "example_sg":
| ...
│ 
│ An argument named "tags" is not expected here.

P.S The code worked fine while the modules were referenced from the local folder.

5

As an update to previous post. I managed to solve the problem that was appearing when we triggered terraform validate and terraform apply. Even though the modules were properly parsed by the HashiCorp API the problem appeared to be the zipping of those modules. Our script somehow zipped the module(s) with their root directory in place (even though the Terraform Cloud private registry parsed and displayed them properly).