In order to give further advice, I think you’d need to share with the forum a bit more detail.
Thanks for your response, i will try my best to go further in detail.
Thanks for linking to the blog article, to explain what you meant by “multi-repo approach”. (It’s not really a standard term outside of that article, in my experience.)
Suprised by that, i have seen it mentioned in a few different places. What is the standard term for this approach?
There’s a colon in an odd place in your S3 URL - is that colon after env supposed to be there?
You said you have a state file for each repo, but the S3 URL, and pictured repository layout seem to contradict this - it implies you have a state file which is:
- specific to each environment (
prod)
- specific to the top level directory within the
infrastructure repo (rds)
You’ve not mentioned how your environments (e.g. prod) are represented in your current setup - please elaborate on that.
The url was copy and pasted directly from aws s3 console, it does contain that :
You are correct our state file is specific to a terraform workspace and the specific top level directory.
I did not think mentioning that we use workspaces was worth it, as to be honest its more of a “habbit” than anything else, after the init process we select a prod workspace terraform workspace select prod.
We use it for “some” of our infrasturcutre to build a dev version of the resource, before prod to test new configurations. But we found workspaces are difficult to deal with when it comes to seperating environments. Its also one of the main attractions i see with “multi-repo” approach.
That might be a good idea - or it might not be. Which it is, depends a great deal on whether this module is a clearly thought out re-usable unit, or just “everything my project needs to do with RDS”.
The tf-module-rds will be a reusable unit to create aws rds instance including the cloudwatch alarms etc all built in terraform.
It also matters quite a bit how you use your prod and dev environments. Do all your changes to prod, first pass through dev, like a flowing pipeline? Or, are there likely to be things in dev which are experiments that will never go to prod? What about emergency changes in prod that don’t necessarily go through dev?
We test configurations in dev before prod. But we rarely do this, i anticipate this becoming more relevant in the future.
What do you anticipate living in this repository? Just a tiny configuration with a single module block identifying a version in the tf-module-rds repo? Or something more complex?
Exactly, i anticipate prod/main.tf to have a single module block identifying the version of tf-module-rds repo. Also having a dev/main.tf to have a module block identifying a different version of ts-module-rds repo. Allows us to tset changes and see a clearer seperation than what we have with using terraform workspaces.
The biggest thing I’d like to emphasise in response to your post, is this is not an all or nothing decision. I fully support the idea of storing re-usable modules that are separately versioned, and included in multiple other configurations, in their own Git repositories. On the other hand, if you have modules that are used exclusively in only one other repository, I would strongly question what anyone seeking to split them out was hoping to gain from that - and have merged Git repositories to undo that sort of separation.
You are correct a lot of these modules will only be used exclusively in only one other repository. But i believe
- having things separately versioned is the main attraction via tagging modules.
- being able to build ci /cd pipelines to automate and verify infrastructure changes other team members are making will be much easier if we have multiple repo’s,
- i can assign users specific permissions to modify one repo but not the other, if everything is togeather, all of our devs will have access to every infrastructure config.