I’m wondering the performance difference between data resources. The environment I’m working on is somewhat of a “Power Terramod” configuration. It currently has lots of data resources which causes plan/applies to take forever. I’m looking for a path forward to make plan/applies go faster.
My understanding is that I can do an external calls (as we are) using a data resource that hits a provider (data.aws_security_group.name.id). Split the env into multiple states and then use remote state (data.terraform_remote_state.sgid). Or consolidate states and extent modules with outputs that give direct pathing to the resource (module.env-sec-groups.sg-id-1)
They each have pros and cons, but I’m currently trying to solve the performance problem. Obviously keeping them all in one state would be the most performant since its all in memory but has other problems. I’m curious what the performance difference between a standard data.aws_security_group.name.id and data.terraform_remote_state.sgid. They are both making an external call to AWS (due to my remote backend in S3), but one has to be faster than the other. Maybe its not much but doing it hundreds of times adds up.
I thought about trying to build some sort of test harness so accurately test but I figured I would ask the community before starting a whole project.