Hi everyone, I hope all of you and your families are okay. I’d like to ask for advice.
I’m currently working in the process to start making Cloud Projects with IaC. We are trying to make terraform modules that accomplish several security policies that have forced us to avoid using public modules and public resources. So we need to create each AWS resource into a module which already implements the necessary security controls, for example into S3 element, we need to create the bucket with public access disabled, encrypted by SSE-S3, the versioning turned on and so on.
We also have modules for principal elements on the accounts that we named “Landing Zones”, for example Networking, Security, Communication, etc. These landing zones are supposed to be implemented on the majority of accounts.
All the projects have the necessity to be purged from all the infra deployed in a spontaneous moment or via scheduled work plan (only having infra on workdays at work hours) to save some money especially on pre prod environments.
In order to that and taking the example of the next project:
A project has a VPC composed of Subnets, RouteTables, Attachment transit gateway. Also in security we have several services such as CloudTrail, GuardDuty, AWS Config, and others. We have private endpoints and NAT Gateways. And for the application infra we’re using ECS with fargates, and RDS for database and EFS for sharing content for the fargates.
Which is the best practice to accomplish a good management of the infra. Taking into consideration that the fargates will be in constant change, growing in number, changing versions of the docker image etc.
We have considered this to possible ways, Have two terraform projects, one for everything that needs to stay deployed on the account and all the other infra that can be purged any moment.
--------------Do not necessary to delete-------------------
├──Landing Zone Networking
│ └── main.tf // contains VPC, subnet, RT, attachment transit gateways
--------------Do not necessary to delete-------------------
--------------Prone to be removed-------------------
├──HUGE_PROJECT
│ └── main.tf
└── modules //REMOTE PRE VALIDATED MODULES
├── Landing Zone Security
│ ├── main.tf // contains CloudTrail, GuardDuty, Config
│ ├── outputs.tf
│ └── variables.tf
├── Landing Zone Communications
│ ├── main.tf // contains NAT Gateways, Private Endpoints
│ ├── outputs.tf
│ └── variables.tf
├── Application
│ ├── main.tf // contains ECS cluster, ECS Service, ECS Task
│ ├── outputs.tf
│ └── variables.tf
├── Database
│ ├── main.tf // contains RDS and all the necessary components
│ ├── outputs.tf
│ └── variables.tf
├── Storage
│ ├── main.tf // EFS
│ ├── outputs.tf
│ └── variables.tf
--------------Prone to be removed-------------------
In this approach we thing we can have only two states files and if we need to remove all the infra we will do it in a quick way. With this approach wi will win a operative facility, cause we can manage the inputs and outputs originated by the terraform modules in the easiest way. But the project can be complex.
And this one:
├──Landing Zone Networking
│ └── main.tf // contains VPC, subnet, RT, attachment transit gateways
│ ├── outputs.tf
│ └── variables.tf
├── Landing Zone Security
│ ├── main.tf // contains CloudTrail, GuardDuty, Config
│ ├── outputs.tf
│ └── variables.tf
├── Landing Zone Communications
│ ├── main.tf // contains NAT Gateways, Private Endpoints
│ ├── outputs.tf
│ └── variables.tf
├── Application
│ ├── main.tf // contains ECS cluster, ECS Service, ECS Task
│ ├── outputs.tf
│ └── variables.tf
├── Database
│ ├── main.tf // contains RDS and all the necessary components
│ ├── outputs.tf
│ └── variables.tf
├── Storage
│ ├── main.tf // EFS
│ ├── outputs.tf
│ └── variables.tf
We’ll have several projects of terraform with several states and when we need to apply changes only affect the particular project as same as the state. With this we win facility on managing the projects ‘cause they are fine-grained projects. But the dependencies in inputs will be satisfied via data sources on Terraform code.
Sorry if this is too much text, but we want to start this terraform era with the right foot.
How do you guys manage similar projects?
Which is the most secure and efficient way to manage these large projects?
Are we doing something wrong?
I’ll be immensely grateful if some of you can text me some advice!
Thanks for reading and greetings from Mexico City.