I’m a software engineer, so when I took on our company’s Terraform project, I looked at it as a software project, rather than a scripting project, that needed to be handed off to other teams. I took a ‘configuration driven development’ approach, where I described the full and final system in the .tfvars file and then built a framework behind it. This allows our delivery teams to build out complex environments without needing to have any knowledge of AWS or Terraform – but they get our best practices baked in.
The attached file, perf.tfvars (at the bottom), is a typical configuration file. It has objects for describing the systems and AMIs needed, but then has an environment variable with an array of ‘subsystems’, which are essentially security groups in AWS (by keeping the names generic, I can keep the same configuration for Azure, etc. and just build out a supporting framework for it). Each subsystem describes the inputs, machines, scaling groups, etc. that it needs.
I then have 20+ modules that walk this structure in turn, pulling out the bits they care about and building them out. By maintaining a careful naming structure for everything, I can keep doing lookups on other modules to get the id/names I need to make it work. I included two modules as samples. One is for security groups, which is very simple. It walks the structure, creates the array of everything that needs to be created and then creates the resources. The other file is the security group rules and shows that it can get pretty salty as it has to pull security group rules from multiple places in the array in various formats. It does multiple lookups on the established security groups to makes sure the rules are assigned properly. Even being one of the more complex modules though, it’s still only 100 lines or so. It all works extremely well with my team, who has a fully defined and understood infrastructure.
The problem I’m running into however is when I need to make modifications to an infrastructure. It seems like Terraform keeps all this information more or less in an array like I create it and if something in the middle of the array needs to be adjusted, everything from there down needs to be destroyed and recreated. So, for example, if I have 10 security groups and need to change a rule in the 4th, Terraform wants to destroy security groups 5 thru 10 as well (and everything related). That tends to upset the other teams to where they don’t want to use this framework (they’re still developing their infrastructure and need to make iterative improvements regularly).
I’d like to know, generally speaking, if the approach I’m using is conducive to being able to modify pieces within the array without destroying the rest of it in the process. For example, I’m wondering if I should query the infrastructure each time for security group resources rather than passing that data from module to module, if that would help. Or maybe this is the wrong approach altogether for modifications and I should start over with a more ‘scripted’ approach.
Hopefully this post is detailed enough to understand what I’m trying to do. I’m just looking for a thumbs up/down on this approach in general from the people that have been working with Terraform longer than I have.