I have been playing with the newly introduced module testing experiment, but I am unable to load a local custom provider. terraform test seems to ignore dev_overrides and providers in ./terraform.d/plugins/.... Is there any other way to load a local custom provider?
Hi @danischm,
I would expect terraform test to not honor dev_overrides today because dev_overrides is a runtime setting rather than a terraform init setting and terraform test is, in effect, running a slight variation of terraform init as part of its work in order to initialize each test suite directory as a root module.
However, I’m surprised to hear that the other installation methods aren’t working. The terraform test command uses the same provider installer that terraform init does, just reconfigured slightly to write providers into a directory under the test suite rather than under the main configuration, and so it should be picking up all of the other provider installation settings that the terraform init command would use.
One way we could try to debug this further is to run terraform test with the environment variable TF_LOG=trace set. On my local machine I tried that on an example configuration which intentionally had a non-existing provider in it and saw it report the following:
2021-05-14T09:19:43.558-0700 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
I then created terraform.d/plugins and saw it report the following instead:
2021-05-14T09:20:18.051-0700 [DEBUG] will search for provider plugins in terraform.d/plugins
So at least on my system it does seem to pick up that directory as one of the search locations.
As a further test I created the expected filesystem mirror directory structure under terraform.d with an empty file standing in for the non-existing provider plugin my configuration called for. Since an empty file isn’t a valid plugin executable I expected to see this fail with a different error indicating that it found the provider but wasn’t able to execute it, and indeed it did:
╷
│ Error: Could not load plugin
│
│
│ Plugins are external binaries that Terraform uses to access and manipulate
│ resources. The configuration provided requires plugins which can't be located,
│ don't satisfy the version constraints, or are otherwise incompatible.
│
│ Terraform automatically discovers provider requirements from your
│ configuration, including providers used in child modules. To see the
│ requirements and constraints, run "terraform providers".
│
│ failed to instantiate provider "registry.terraform.io/hashicorp/boop" to obtain schema: fork/exec
│ tests/foo/.terraform/providers/registry.terraform.io/hashicorp/boop/1.0.0/linux_amd64/terraform-provider-boop: permission denied
│
│ Plugin reinitialization required. Please address the above error(s) and run
│ "terraform init".
│
╵
This message is of course rather unfortunate because it’s recommending to run terraform init even though that isn’t actually relevant to terraform test, but that’s one of the limitations of our current experimental implementation approach – Terraform Core doesn’t actually know it’s being run in a slightly different way for testing here. That aside, this does indicate that it found and couldn’t execute my provider plugin; it was because I didn’t give it executable permissions rather than because it was invalid, but either way it found the local plugin and skipped trying to install it from registry.
I’d be interested to see what happens if you try to follow these same debugging steps on your end and see if you run into a different result at any point.
Thanks a lot for the detailed explanation! It was indeed my mistake, as I still had dev_overrides in my configuration file which replaced the implied configuration. But it looks like I ran into the next issue with variable values defined with environment variables (TF_VAR_*) being ignored. Is this expected?
Hi @danischm,
The intention for variables is that your test configurations in tests/ will provide the input variables your module expects for each test case, with the tests all being self-contained and repeatable. There isn’t any mechanism to set variables on a per-run basis.
Hi @apparentlymart ,
In my case the provider being used in the test configuration requires credentials to be configured in the provider section. The idea was to inject those through environment variables when tests are being executed in a CI environment. I know there are other/better ways to do this, but it seems in this case I might be limited by what the provider supports.
Hi @danischm,
In a typical provider design it’s a last resort to pass credentials directly into the provider configuration, because providers should also offer out-of-band credentials mechanisms that are conventional for the vendor in question, such as the AWS provider understanding ~/.aws/credentials files and the conventional AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY environment variables. For a provider designed in this way, the credentials-related arguments in the provider configuration are for the relatively-rare situation where a particular configuration needs to use multiple sets of credentials at once, and thus the provider needs help to choose the correct credentials for each one.
Since you’re working with a provider you’ve created, I guess you might not have designed it in this way. I would suggest considering following the usual provider design pattern I just described if you can, even if that just means defining some special environment variables that the provider reads directly using os.GetEnv (assuming the system in question doesn’t have some existing convention to mimic).
With that design, you can limit your module inputs only to what’s relevant to what you are testing and keep all of the administrative credentials stuff out-of-band, and provided by your CI system in the case where you are running in automation as you suggested.