Troubleshooting Slowdown

Hi Community,

i’ve been using terraform for the past four years, and love it. Thank you for all your work. I’ve contributed back to a few bugs and even learned a bit of go to help contribute some code.

Unfortunately I’ve hit a new ‘operational’ bug - and need some help in working out the process I should go through to troubleshoot.

I am using workspaces, with about 12 in my configuration. What I’ve seen over the past couple of weeks is a significant degradation in the time it takes terraform to execute a plan. It used to complete in under a couple of minutes, whereas now it can take up to 15, but most of the time executes around 8. There are quite a few resources in the configuration, with the majority of components wrapped into their own modules.

I’m aware of TF_LOG to try and see what’s happening under the skin, as well as -parallelism to try and speed up the pace at which the graph is traversed after a refresh.

I don’t know whether it was a recent change made to introduce a module for WAF2, or whether it’s the fact that the module is shared across all 7 API Gateways in the configuration that is the cause of the slowdown.

Can anyone recommend how I should be troubleshooting this and what I should be looking for? When I run with TF_LOG, there doesn’t appear to be any slowdown on the terraform refresh; but on the stage just before the plan is printed out.

Once I’ve solved this one, I expect to put up a blog post to help others.

Thanks in advance,

Andy

Hey Andy,
Most probably it might be connected with WAFv2 introduction, check here: https://github.com/terraform-providers/terraform-provider-aws/issues/14062