It seems that enabling provider caching automatically triggers the “Incomplete lock file information for providers” warning when you run a terraform init. Why do I have to download the provider every time to “be safe?”
maxb
March 6, 2023, 3:25pm
2
You don’t - it is safe to ignore the warning.
Further background information can be found at
and
opened 04:17PM - 11 Dec 20 UTC
enhancement
cli
### Current Terraform Version
```
$ terraform version
Terraform v0.14.2
``… `
### Use-cases
I have 200+ root modules and I'm automating a provider version up workflow in CI with [tfupdate](https://github.com/minamijoyo/tfupdate), which updates all version constraints in Terraform configurations recursively. My laptop is macOS and CI is Linux, so I want to pre-populate hash values for all platforms I need in the workflow to avoid a checksum mismatch error.
I'm looking for an efficient way to maintain `.terraform.lock.hcl` for multiple root modules and platforms environments.
### Attempted Solutions
For downloading providers:
As you know, there are two hash formats, that is, zh and h1, and zh is recorded only when a provider zip package is downloaded from Terraform Registry. I want to avoid redundant downloads because I have a lot of root modules, so I tried to create a local filesystem mirror with the `terraform providers mirror` command at git project repository root, and then generated lock files at each sub directory with the `terraform providers lock` command using the mirror. It recorded only h1 hash values. It's ok.
I wrote a script to generate `.terraform.lock.hcl` for multiple directories and platforms.
```bash
#!/bin/bash
set -eo pipefail
# create a plugin cache dir
export TF_PLUGIN_CACHE_DIR="/tmp/terraform.d/plugin-cache"
mkdir -p "${TF_PLUGIN_CACHE_DIR}"
# create a local filesystem mirror to avoid duplicate downloads
FS_MIRROR="/tmp/terraform.d/plugins"
terraform providers mirror -platform=linux_amd64 -platform=darwin_amd64 "${FS_MIRROR}"
# update the lock file
ALL_DIRS=$(find . -type f -name '*.tf' | xargs -I {} dirname {} | sort | uniq | grep -v 'modules/')
for dir in ${ALL_DIRS}
do
pushd "$dir"
# always create a new lock to avoid duplicate downloads by terraoform init -upgrade
rm -f .terraform.lock.hcl
# get modules to detect provider dependencies inside module
terraform init -input=false -no-color -backend=false -plugin-dir="${FS_MIRROR}"
# remove a temporary lock file to avoid a checksum mismatch error
rm -f .terraform.lock.hcl
# generate h1 hashes for all platforms you need
# recording zh hashes requires to download from origin, so we intentionally ignore them.
terraform providers lock -fs-mirror="${FS_MIRROR}" -platform=linux_amd64 -platform=darwin_amd64
# clean up
rm -rf .terraform
popd
done
```
However, with the lock file recorded only h1 hash values, if I run `terraform init` without any mirror or cache, the init command adds zh hash values to the lock file. It causes an unexpected lock file change.
I expect `terraform init` without the `-upgrade` flag not to update the existing lock file, because `terraform init` is essential command for all workflows and I think it's not desirable to cause a git diff unexpectedly. Is it intentional by design or a bug?
Pre-populating zh hash values requires redundant downloads, so I want to avoid it. My pain points are:
- Recording zh hash values always requires download from the registry.
- The registry doesn't return h1 hash values (Is it right?), so I need to create a local mirror to calculate h1 hash.
If my understanding is correct, it would be great if the Terraform Registry returns not only zh hash values, but also h1 hash values for all platforms without download.
For downloading modules:
If a root module depends on other modules, the `terraform providers lock` command requires `terraform init`. I understand selecting a correct version needs all module contents for completeness.
However, before Terraform v0.14, there was no lock file, so I've already defined all provider dependencies I need in the `required_providers` block at each root module. In that case, it's not always necessary to get all modules because I know it. It works but it's inefficient. It would be great if we can have a shallow check option at own risk.
### Proposal
Allow the `terraform providers lock` command to generate `.terraform.lock.hcl` including zh and h1 hash values for given platforms from `required_providers` block without downloading providers and modules.
### References
Related to:
- https://github.com/minamijoyo/tfupdate/issues/32
- https://github.com/hashicorp/terraform/issues/27161
In the forthcoming v1.4.0 release Terraform will construct a complete dependency lock file by default, so you can check that into your version control and avoid encountering this situation.
Even in current versions of Terraform (v1.3 and earlier) I would expect this message to appear only when Terraform adds a new entry to the lock file. Once you already have the file in your version control system you should not see the warning again until you next upgrade the provider, which will then of course regenerate the lock file entry and so you’ll once again have an incomplete entry that needs to be amended with checksums for other platforms.