What Terraform is missing?
Terraform is great, but it’s missing a couple of features to make it great.
In the real of devops, the environment is mixed. Some cloud services are created with Terraform using a CICD process. Other times, a cloud service might be created manually or even with Terraform but outside of CICD process (different configuration file). The real world of devops is a mixture of both types of ways of creating cloud sevices, as new requirements creep-in or the tech team needs to try out a small deployment of a few instances…etc.
Terraform does not address the cross-reference of cloud services component in a satisfactory way, although Terraform tries to by using certain constructs such as … These cross-referencing solutions are not elegant, confusing at times and difficult to use. Compounded with the previous point, if a cloud service is created and needs to reference another cloud service component that might have been created manually or with Terraform but on different configuration file, then the only way is to look-up the component-id or the Arn.
Terraform can do better than that. Terraform needs to create a universial web service component cross-referencing that is agnostic to the way the component was created (manullay or with Terraform or some other some other utility). This cross-referencing system must be simple, obvious and easy to use.
Let me outline what this system might look like.
• Most cloud system support tags on all their components. That’s key to the solution.
• Terraform can create standard tags with which it can run queries. For example, each created component could have these tags: TF_COMPONENT=’some name’, TF_COMPONENT_ID=’some unique number or code’
• Terraform can provide a library of functions to retrofit cloud components that don’t have these tags. This way, the user can upgrade their manullay created cloud component to allow for cross-referencing.
• Terraform can provide a series of library functions to get the id or arn of a component that has TF_COMPONENT. The user needs only to provide the name that comes with this tag.
• Naturally, each cloud component created with Terraform will have these tags.
• Using the cross referencing system should be simple and obvious, like it’s done in programming languages like Python. Like the following lines below. These are import functions on already existing cloud components and services.
◦ ‘subnet_id = import_subnet_id(tf_component=’subnet_private_1’)’
◦ ‘security_group_id = import_security_group_id(tf_component=’allow_ssh_sg’)’
◦ ‘my_keypair_id = import_keypair_id(tf_component=’team_kp’)’
◦ ‘bastion_ec2_id = import_instance_id(tf_component=’bastion_instance’)’
• Other more complex functions should also be provided for more complex import queries.
◦ ‘subnet_id = import_vpc_subnet(vpc_tf_component=’service_vpc’, tf_subnet_component=’subnet_public_1’)’
This is the beginning of a solution and needs to be further analysed and defined.
The benefit of such universal cross-referencing system would be enormous. I will explain why.
1. Many devops operators like me, no longer believe in a single Terraform infrastructure configuration to create the whole system. It would be much better to have a modular approach for Terraform configuration files, where each Terraform component can also be created separately from the rest of the system. This is what the work environment requires as new orders come from the top even during a project. Often times, only a small part of the infrastructure needs to be re-created, replaced or duplicated for whatever reason.
2. Cloud components could be created seperately or in small groups or with the rest of the system. The devops needs that level of flexibility. When a cloud component sub-systems are created, a master configuration file “calls out” the individual component configuration files with proper cross-referencing technology.
3. The advantages of such system are obvious and would go a long way to satisfy the needs of the devops operator.