Hey, I want to run terraform scripts in multiple docker containers parallelly. Can anyone of you suggest to me how to do that? I tried through docker-compose & Dockerfile but docker terraform images don’t support /bin/sh.
I think you’ll have to explain in more detail what you’re trying to do, if you want a useful answer. Currently so much is unsaid, I don’t really have any idea what your goal is.
so my goal is to execute the multiple containers containing the terraform file with one command. Like, I am able to execute the terraform files individually in a separate docker container. Now, I want to run them parallelly which I am unable to do.
I am unable to follow your explanations. Possibly have a read through Guide to asking for help in this forum
You don’t “run” individual Terraform files, you just run Terraform against root modules. Each root module has its own separate state (either a local file or something remote such as an S3 bucket).
So you can restructure your code into multiple root modules, each with a separate state, and then they can be run independently (in parallel at the same time if you wish). Note that this is just running multiple instances of Terraform so you’d need to separately check each plan output or approve an apply (rather than something you consolidated) and also passing of values between root modules would need to be managed (for example via the remote state data source, which then adds complexity around ordering of runs).
What are you actually trying to solve? Is it that your Terraform run is taking too long? Terraform will already run things in parallel (by default up to 10 at once): Command: apply | Terraform | HashiCorp Developer
Take a scenario there are terraform1 folder and terraform2 folder and each folder contains terraform configuration files. Now, I want to run these two in the docker container parallelly, which I cannot do. I made Dockerfile for both folders and converted it to an image now if going to run them parallelly then get an error that Terraform doesn’t support either /bin/sh(I include it as a command) or multiple commands (Terraform init, plan apply, etc.).
Normally, we know we can achieve parallel execution of terraform configuration files in a separate folder either through Python or scripting or terragrunt. But, how to achieve it through docker
You haven’t included the Dockerfile you are trying to use or much in the way of details, but you can include anything you want in your Docker image. For example you can create a Dockerfile based on Debian and then install Python, Terragrunt, Terraform, etc. as needed and/or include a shell script that runs the commands you are wanting. If you are trying to use an existing image that already has Terraform installed you’d need to figure out what else is there and if it supports easily adding additional software (e.g. via apt-get).
However I’m not really clear about what you are trying to actually do. Docker-compose is a method for running multiple Docker containers, generally a set of longer lived non-interactive applications. It isn’t really a good fit for containers that run for short periods or need checking/interacting (such as saying “yes” to approve an apply).
It sounds more like what you might want is either some form of wrapper application around Terraform (such as Terragrunt) that allows someone to work with multiple root modules at once (which could be delivered as a single Docker image if wanted), or something like a CI/CD/job scheduling system (Jenkins, GitHub Actions, Celery, etc.)
From hashicorp/terraform:1.5.0
WORKDIR /terraform1
COPY . /terraform1
CMD [“/bin/sh”,“-c”,“terraform init; terraform plan; terraform apply”]
My Dockerfile for one of the folder.
version: ‘3.8’
services:
terraform1:
build:
context: ./terraform1
dockerfile: Dockerfile
volumes:
- ./terraform1:/terraform1
terraform2:
build:
context: ./terraform2
dockerfile: Dockerfile
volumes:
- ./terraform2:/terraform2
my docker-compose file
My suggestion: Stop using Docker here - just run Terraform directly. Layer of complexity removed, and problem solved.