I’m sorry if I’m misunderstanding your framing here, but it seems like you’re trying to go from one huge
main.tf file to using multiple separate directories without considering the less extreme solution of having multiple
.tf files in the same directory.
In case you’re not aware of this, Terraform doesn’t actually distinguish between the various
.tf filenames and so there’s no reason why you need to put all of your resources in a file called
main.tf; that’s just a convention that emerged for simpler modules that are focused enough in a particular problem that there’s no meaningful way to distinguish different components.
How to split things into multiple files is ultimately up to you; in different teams I’ve seen different strategies but I think the two most common ones are:
- A separate file for each separate “service” or service-like structure the cloud vendor offers. For example, AWS users typically end up with files like
rds.tf to group their resource declarations by roughly the same boundaries as they’d be grouped by in the AWS console, thus making it easier to translate between what you see in the console and what you see in the Terraform module.
- A separate file for each conceptual layer/subsystem of the overall system, as viewed by the team building the system. This one is harder to give a good example of because it inevitably depends on how your system is built, but a generic sort of version of it might be files like
storage.tf for the data tier,
web.tf for the web server and load balancer tiers, and
worker.tf for a generic worker tier, if your system is one that makes distinctions of that sort.
Terraform considers all of the
.tf files in a particular directory to belong to the same module, so you shouldn’t need to do anything except cut and paste the configurations you already have in
main.tf into other files. Terraform should see the result as exactly equivalent to what you started with.
As you’ve seen, Terraform uses separate directories as the representation of modules, and so the requirements for separate directories are a little harder: you need to explicitly define the interfaces between the modules using input variables and output values. But given that the problem you started with was a sense that the
main.tf file is too big, I’d suggest starting by just splitting it into multiple files in the same module and then see over time if there emerges a more explicit architectural boundary that would make the additional overhead of a module abstraction worthwhile.