I have a terraform folder containing multiple “resource type - groupname.tf” files to deploy Azure networking, VMs, NSGs, load balancers, etc, with the terraform state stored in an Azure storage account. There are a lot of instances of “create X resource in one file, then use X resource ID / other attribute in another file” (e.g. create subnet, use subnet id in the VM file).
The problem i would like to solve is that making a small change to an NSG ruleset currently requires me to run the whole file set through terraform, which takes 3+ minutes of working through the entire environment to determine that it just needs to change a couple of NSG rules. This would also reduce risk by not having terraform touch & potentially break unmodified items (e.g. the recent problems with azure recovery services vaults & API capitalisation).
One option would be to create separate folders and tfstate files for some resource categories or groups (e.g. all the subnets), add output variables for the attributes I’ll need (e.g. subnet names, ids, rgs etc in the subnets example). then to use “remote_state” and “data” items in successive folders (e.g. NSGs, VMs) to pull that info into to use it in assignments. This would require more in-depth understanding of how the project hangs together on the part of anyone making changes to it. (plus it will require some manual changes to the current state file).
Is there a better way? if yes, can you point me to some worked examples or guides?
Thanks in advance,