Which Abstraction model is recommended?

I am looking for some guidance on what abstraction model is good for my scenario.

We have created architecture patterns for our cloud solutions like
2-tier application with web and database
micro-services applications on Kubernetes with different data storage backends
Data analytics ETL pipeline with Data lake storage and Databricks

We are planning to provide infrastructure provisioning for these patterns using Terraform Our hope is that, our customers can directly provision the pattern that they find it suits their business use case.

What would be the best way to organize the Terraform configurations for these patterns?

Option 1: We breakdown configurations into independent modules for each resource type and then define the patterns as module composition with placeholders for corresponding variables.

Option2: We create each pattern as a module with all resources that are contained in the pattern. One concern is the repetition of resources that can be present across patterns.

Or is there any other better option? Thanks for any inputs.

@mnltejaswini medium-sized project layout works best in 90% of the cases - Medium-size infrastructure with Terraform - Terraform Best Practices

Option2: We create each pattern as a module with all resources that are contained in the pattern. One concern is the repetition of resources that can be present across patterns.

this is definitely the way to go. Option 1 is rather an anti-pattern because you’ll never manage to catch up with resource new features.

Hi @mnltejaswini,

If I’m understanding correctly the intent of each of your options, it seems to me like these two options are not necessarily in conflict with one another.

By that I mean: if you follow a module composition approach where the problem is broken down into small parts that can be combined in different ways, you can still potentially write wrapping modules that combine the smaller modules together to represent commonly-used patterns at a higher level of abstraction.

It is true that the module composition guide recommends having only one level of nested module, but ultimately that is only a guideline. As long as each module is representing a relatively-clearly-scoped abstraction, there isn’t any significant problem with building a higher-level abstraction in terms of lower-level abstractions, within reason.

While it’s impossible to give specific advice that would apply to all situations, I think if I were in the same situation I’d start with the “option 1” module composition approach at first, but be mindful of how those smaller modules are being used and try at first to minimize the number of distinct combinations of the modules. In other words, to initially standardize by convention and code review rather than by sharing code. In doing that, I expect you’ll start to notice particular combinations that arise repeatedly and so you can consider the tradeoff of bundling those together into a higher-level module.

I think I would typically draw the line at two levels of nesting, though… not for any particular technical reason, but just because each level of nesting creates another possible set of combinations of the lowest-level building blocks, and so it’ll be more likely that you run into situations where the higher-level abstractions start overlapping with one another in ways that make them hard to compose together.

I hope that helps!