Upto version 0.12.x I used to be able to “cook” a Terraform directory with the binary and the providers together in the same directory (it being updated from time to time).
Now, with version 0.13.5, it always wants to download the version in the current directory, which ends up downloading ~ 100+ MB.
i.e.
Q: For version 0.13.5, what is the directory layout I would need for the plugins to have a single unified directory of ‘terraform binary + plugins’ ?
Assuming my terraform binary is located at /opt/terraform-0.13/bin/terraform, where would I need to place the plugins?
After some experimentation, I realize that if you specify the -plugin-dir to init, it remembers it in the file ./.terraform/plugin_path.
Also, after setting TF_LOG=debug and running terraform init the directories Terraform is looking up is visible in logs.
Looks like Terraform has stopped using the “directory of the binary” as one of the lookup directories ?!?!
This is what I was asking in the Office Hours @pkolyvas @mildwonkey
Hi @shantanugadgil,
From Terraform v0.13 onwards providers need to be in a multi-level directory structure rather than just a single directory, because the provider namespace is now hierarchical rather than flat. As a consequence, there isn’t a direct equivalent to your previous model of placing the terraform executable and the terraform-provider-* files all together in a single directory, because that wouldn’t give Terraform enough information to know which provider each of the executables is intended to represent.
The most direct way to translate what you were doing before would be to put the provider plugins in one of the implied filesystem mirror directories and then place the terraform executable in a directory alongside the plugins directory so that they are all together.
With that said, there shouldn’t be any particular need to bundle a Terraform executable with particular provider versions, because Terraform is able to query a directory containing multiple versions of a provider and select one of them. A typical approach is to create a single directory containing all of the provider versions you need and then share that directory between multiple Terraform versions. Because the directory layout changed between v0.12 and v0.13 this does mean that supporting both at once will require creating two directory layouts, but that situation is specific to that particular pair of versions.
You also implied in your question that avoiding downloading lots of large packages repeatedly was part of your motivation for setting up this local mirror of various providers. If that’s true, you might find it more convenient to activate a provider plugin cache directory, which is a directory Terraform will download packages into so that it can potentially re-use them on future terraform init, as long as the versions in the cache are still the versions the current configuration requires. This is a popular option when the main motivation is avoiding re-downloading the plugins, because it doesn’t require any special effort when one of your Terraform configurations needs a new provider version: terraform init will download it just that one time and write it into the cache for future use.
I hope one of these options is helpful to adapt your previous approach. There are some other options too, which are described in detail under Provider Installation in the CLI Configuration documentation. If you see something else there that seems more appropriate than my suggestions above but you have questions about it, please let me know and I’ll try my best to answer.
1 Like
I will read up the docs to see if any of the existing solution will just work 
to mimic the previous use case.
This “portable TF” idea was a side effect of always trying to use the latest GA versions of the providers as well as the binary.
(The occasional hit caused by some minor bug was deemed fine in comparison to having the versions embedded in the .tf files for long durations and then doing sudden jumps in the provider versions.
(catch any possible errors as soon as possible, was the thought)