Terraform plan taking long time

Terraform
1

Hi,

We are using terraform version 0.12.29. We have a pipeline which runs the following commands:

        terraform init
          terraform validate
          terraform plan
          terraform apply -auto-approve

The time it used to take in the beginning was nearly 5 to 10 minutes but now as our codebase is increasing day by day and the number of resources are increasing the time the pipeline takes normally now is 30 minutes to 35 minutes and we are growing continuously hence, the time to deploy is becoming a hurdle.

We are using Datadog provider but I don’t think so that might be the issue.

1 Like
2

Hi @aujjwal-smarsh,

Without any more details it’s hard to offer specific advice. One option you have is to test higher settings of the -parallelism flag. The default is 10, but if your configuration contains a lot of resources that can be concurrently updated and the remote API allows it, increasing that can speed things up considerably.

A lot has changed since v0.12 too, so if you are running into internal limitations within terraform, upgrading to a current release may provide some extra benefit too.

3

Thanks @jbardin for the suggestion. I used parallelism but it doesn’t show much difference. It used to take 30 min before but after adding parallelism 20 it takes 29 min 40 sec, parallelism 30 is same around 30 min and parallelism 40 is 31 minutes.
Do you know why it is increasing as we increase the parallelism after certain level?

Secondly, I find the refresh state holds the large chunk of time. I am using a Datadog provider. Does making refresh=false be a good option?

Thanks

4

A variation of around 3% when making so many network calls to remote APIs isn’t really conclusive of anything. Again, without specifics I can’t say for sure, it’s possible the structure of the configuration doesn’t allow for much concurrency. Many services have rate limits for protection too that are easy to hit with large configurations, so you may only be able access so many resources within a certain amount of time.

If significant time is spent reading the remote resources, then -refresh=false could speed things up by skipping the first read. That may run into other problems however if resources aren’t correctly updated before planning (and in old versions of terraform, data sources may behave unexpectedly as well).

5

Another thing to think about is splitting into multiple root modules. Often there will be logical groupings of resources which might be owned by different groups or have different change cadences. It can make a lot of sense to split things instead of having one root module that does everything.

6

Hi @stuart-c ,
Thanks for the suggestion. Sorry, I am quite new to terraform and have less understanding of modules. Is there any resource I can refer to understand what you meant by root modules and by different groups and change cadences.

Regards

7

Thanks @jbardin

I have got a better insight into what things to consider.

Regards

8

A root module is the place you run the terraform command, and has an associated state file.

So multiple root modules means having different places that Terraform is run, each with their own separate state file.

Exactly how you structure this would be up to you, but common options include different git repos, different directories, state files in different S3 buckets or paths, etc.

9

Got it. Thanks @stuart-c

10

A post was split to a new topic: Terraform plan takes a long time