Module performance

Terraform
1

:wave:t2:

I have a question about Terraform modules: would using modules make running a plan/apply more performant when compared to a single main.tf configuration file?

To provide some context:

I have a customer using our Terraform provider. They have a single main.tf containing all the configurations for 200 ā€œservicesā€, and each service resource contains nested blocks (e.g. a service might have a domain block or a backend block etc for configuring a web service).

One such resource is a key/value data store which they can seed with values. Now the customerā€™s approach is to use a single configuration file along with for_each to generate all 200 ā€˜serviceā€™ resources and also the nested object store key/values. They pass a large JSON variable which contains the data for the services.

They find a change to a single key/value takes over 4 minutes, whereas I was able to (with a ā€˜module structureā€™ where I had 200 separate modules that the root main.tf module would import) update ALL the key/values for each module (all 200 of them) in just over 3 minutes.

I donā€™t know the customerā€™s full configuration but it does seem like a single main.tf configuration file with 200 resources is remarkably slower to plan/apply than defining 200 separate modules that are imported into the root main.tf module.

Is any one here familiar with how Terraform internally handles things and whether what Iā€™m seeing as far as the time difference between architecture/structure is grounded in any kind of reality?

Thanks!

2

Hi @Integralist,

A Terraform configuration is always processed in its entirety. Modules are a tool to organize, and ā€œmodularizeā€ the configuration for easier development and maintenance, but at runtime they are part of the same configuration.

Itā€™s hard to say what the difference may have been between the configurations you were comparing. In the process of changing the structure, you may have removed unnecessary dependencies, and helped with the runtime slightly. The bulk of time required to plan large numbers of resources however is usually the time it takes to read them all from the remote APIs, and reading 200 resources should take the same amount of time regardless of how they are addressed within the Terraform configuration.