But how can I make sure that my main is connected to these variable files in such a way that when i go into the dev directory i can do a terraform apply command and the infrastructure gets created in the dev enviroment same thing with production. I would really appreciate some more insight into this problem
Before this I tried to work with terraform workspaces however it uses the same backand for all enviroments which reduced the flexibility i needed`
You need three backticks ``` on a line by themselves to enter/exit code block formatting.
It is difficult to answer with precision, as even though you have shown a directory tree, you appear to have left various files out … it is unclear in which directory the Terraform code you have shown us is located, and what filenames your variable files have.
If you place all of the declarations that should be common across both environments in the my_terraform_module directory then your dev and prod directories will each contain only the parts which can vary between environments:
The four values you indicated you wanted to place in variables files.
The provider configuration.
Your backend configuration (not shown in your example, but if you had one it would go here)
Any resources that are unique to a particular environment, which I assume is true for aws_instance since in your example you showed it as being in the root module.
Input variables are primarily for values coming from outside of a Terraform configuration, or that which need to vary from one run to the next. If you already have a separate root module for each environment anyway then a separate variables file is unnecessary complexity, because each root module can directly encode the settings specific to that environment, and delegate to your shared module for the parts which are common across both.
If you still want to use variables files then one way to achieve that would be to place a terraform.tfvars file under each of the dev and prod directories. Terraform will automatically load a variables file of that name when you run terraform plan or terraform apply. That convention is primarily intended for use in automation scenarios (for example, Terraform Cloud automatically generates a terraform.tfvars file with the workspace stored variable settings before running terraform plan) but you can choose to place that file under your version control if you like.