Hi,
I am working on optimizing our terraform pipelines in Gitlab. One approach is to retain the .terraform directory between jobs to avoid having to reinitialize in every single job. However, this has unearthed a weird behaviour.
Our providers.tf
looks like this:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
backend "s3" {}
}
When I run terraform init
locally, this results in terraform downloading version 5.24.0
of registry.terraform.io/hashicorp/aws
. But when the exact same command is executed in Gitlab, it downloads all of the following versions: 4.67.0
, 5.15.0
, 5.16.0
, 5.16.1
, 5.16.2
, 5.17.0
, 5.19.0
, 5.20.0
, 5.20.1
, 5.22.0
, 5.23.0
, 5.23.1
, 5.24.0
. The result are long download times and .terraform/providers
being a whopping 1.1GB in size.
We use terraform 1.6.3 in both cases. Can anyone tell me what is going on here?
I came up with 2 solutions:
- Change the provider block:
version = “~> 5.0” to
version = “YOUR_SPECIFIC_VERSION”
- Download the provider zip file directly from releases.hashicorp.com to your Terraform environment. And unzip it to the standard terraform provider path:
For example, for AMD64 Linux environment, you can place the provider binary to:
/usr/share/terraform/plugins/registry.terraform.io/hashicorp/aws/YOUR_SPECIFIC_VERSION/linux_amd64/
You may need to manually make the directory above. Please make sure that you use the correct version code (e.g. 4.64.0)
In this way, terraform will init with the local binary rather than downloading from public registry.
Hope my comment help.
Hi @christopher.klinge,
Terraform can select and install only one version of each distinct provider; I’m not aware of any logic in terraform init
that would select and download multiple versions.
Therefore I would guess that this is being done by something other than Terraform CLI, but to try to confirm that can you share the full output written by terraform init
while it’s running? It should announce each provider package it’s fetching, so if Terraform is the one doing it then I’d expect to see multiple announcements for the same provider.
Even if it’s not actually terraform init
doing it, seeing the output might give a clue as to what else is doing it.
Hi @apparentlymart,
the output on init
looks fine to me. According to it, only 5.24.0 should have been downloaded. However, it downloaded several versions from 4.67.0 up to 5.24.0.
We do not have any other mechanism in place that would download stuff to those internal directories of Terraform. I wouldn’t know how to download the packages manually in the first place.
Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/local...
- Finding hashicorp/aws versions matching ">= 4.0.0, ~> 5.0"...
- Finding latest version of hashicorp/template...
- Finding latest version of hashicorp/external...
- Finding massdriver-cloud/jq versions matching "0.2.1"...
- Installing hashicorp/null v3.2.1...
- Installed hashicorp/null v3.2.1 (signed by HashiCorp)
- Installing hashicorp/local v2.4.0...
- Installed hashicorp/local v2.4.0 (signed by HashiCorp)
- Installing hashicorp/aws v5.24.0...
- Installed hashicorp/aws v5.24.0 (signed by HashiCorp)
- Installing hashicorp/template v2.2.0...
- Installed hashicorp/template v2.2.0 (signed by HashiCorp)
- Installing hashicorp/external v2.3.1...
- Installed hashicorp/external v2.3.1 (signed by HashiCorp)
- Installing massdriver-cloud/jq v0.2.1...
- Installed massdriver-cloud/jq v0.2.1 (self-signed, key ID 6758F1FC1D4FE114)
Hi @christopher.klinge,
In your earlier message you suggested that you are preserving the .terraform/providers
cache directory between runs.
While I can’t explain how you have ended up with other versions in there in the first place, since I cannot see the full history of your environment, perhaps you could try invalidating that cache to reset back to an empty starting point and then check whether a single run of your job reproduces the large collection of provider packages.
If so, we can try to figure out why that is, but I’ll need to know significantly more about how your automation is set up to know what to consider next.
If your first run after resetting the cache only populates one package per provider but yet you accumulate more over time somehow then we can try to debug that too; in that case I would guess that somehow your dependency lock file isn’t visible to Terraform when it’s running in your automation and so Terraform is treating each run as a fresh start, potentially then selecting newer versions of each provider. (I notice that your terraform init
output doesn’t mention any previous version selections, and so this theory seems most plausible based on what you’ve shared so far.)