How to tell cdktf to use a provider I downloaded on my machine

Terraform CDK for Terraform
1

My goal is simple here:

Because of proxy limitations, I must use local provider - azurerm. I don’t want cdktf to connect to hashicorp servers and download the provider.

Here are the steps I did - after digging deep into stackoverflow questions/answers :

  1. Download the azurerm provider at Terraform-Provider-Azurerm Versions | HashiCorp Releases - I used version 3.41.0

  2. Put the zip file in directory : C:\Users\Rmuf003.terraform.d\plugins\registry.terraform.io\hashicorp\azurerm\3.41.0\windows_amd64

  3. Create terraform.rc file - i’m on windows machine - at %appdata%\terraform.rc i checked the value of appdata with powershell as stated in the docs.

  4. Content of terraform.rc file

    terraform {
    provider_installation {
    filesystem_mirror {
    path = “/Users/Rmuf003/.terraform.d/plugins”
    }
    }
    }

  5. Cdktf init with typescript.

  6. My cdktf.json file :

{
“language”: “typescript”,
“app”: “npx ts-node main.ts”,
“projectId”: “9f9a3db6-92af-4410-91ad-67f04863a5e0”,
“sendCrashReports”: “false”,
“terraformProviders”: [
{
“name”: “azurerm”,
“source”: “Terraform Registry”,
“version”: “~> 3.41.0”
}
],
“terraformModules”: ,
“context”: {
“excludeStackIdFromLogicalIds”: “true”,
“allowSepCharsInLogicalIds”: “true”
}
}

  1. When I type cdktf get or cdktf deploy it always try to connect to hashicorp servers.

Any ideas here ? Reading the docs gives only partial information.

Thanks in advance

2

Sorry for terrible formatting. Here is the content of terraform.rc file :
image

And the cdktf.json file :
image

And the error message i have when i type cdktf get or cdktf deploy :

image
image822×289 9.68 KB

3

Hi @francois.munger :wave:

This is an interesting problem and I think it will take a bit of trial and error to get the behavior you want as this is currently not supported by the CDKTF CLI directly.

There are two separate aspects to it, so I’ll talk about them separately.

cdktf get

The get command generates TypeScript bindings based on the Terraform provider schema JSON. It obtains this schema by creating a temporary directory with a dummy Terraform project using the providers specified in cdktf.json > terraformProviders which it then first initializes to download the providers (running terraform init) and it then invokes terraform providers schema -json to obtain the schema.

Here, I expect the terraform init command to be the culprit that makes calls to the Terraform Registry, downloading the provider and checking its checksums. It seems that you’ve already set-up your .terraformrc fetches that provider from your local mirror instead. I suspect that this step requires access to the Registry in order to fetch the checksums for your provider. Unfortunately, there doesn’t seem to be a way to tell init to not fetch checksums.

Which is why I suspect, that another way forward might be required here. One possibility that comes to mind, would be to use a privately hosted Terraform registry that can be accessed internally and which can serve checksums.

Another alternative would be to get around using the get command by vendoring the pre-built package we have for the AzureRM. Theoretically, you could just take its src/ directory and put it into .gen/providers/azurerm (or anywhere else in your project).

cdktf plan / deploy

Both of those commands invoke cdktf synth which runs your CDKTF program to create the cdktf.out/stacks/<xy>/cdk.tf.json files that contain the code that Terraform is invoked on in the end. Both of those also invoke terraform init and terraform providers lock both of which might make calls to the Terraform Registry for checksums that you might not be able to suppress.
That said, it is possible to only run cdktf synth and then use the Terraform CLI directly in the outputs directory which would give you full control over the commands you invoke (i.e. never invoking the providers lock command).

However, this still leaves you with the problem of terraform init trying to access the Registry for checksums. I’m not familiar with the exact process there, but maybe @apparentlymart can shed some insights here. My best guess would be to try using a private registry (which might even work locally / served from the filesystem).

Maybe Martin also knows about some similar requests (which I’m sure there were) or even docs about running Terraform in environments that don’t have internet access.

– Ansgar

4

Thanks for your quick reply and detailed explanations :slight_smile:

One possibility that comes to mind, would be to use a privately hosted Terraform registry that can be accessed internally and which can serve checksums.

If I use Artifactory (which we have) to store the azurerm provider and then reference it in my cdktf.json file, will it solve the problem entirely ?
If it’s the case, is there a guide that shows how to do this ? My knowledge is very limited with Artifactory, any help would be appreciated.

Regards,
Francois

5

@francois.munger Can you try


cd cdktf.out\stacks\<stack-name>\
terraform providers lock -platform=windows_amd64 -platform=linux_amd64 -fs-mirror=C:\Users\Rmuf003.terraform.d\plugins\plugins

and then run

cd ..\..\..\

cdktf deploy
6

Here is the result :

image
image1023×428 13.5 KB

Additional details :
I left my cdktf.json file with no provider and used the command npm install @cdktf/provider-azurerm
image

My stack :

image
image738×975 42 KB

The lock file :
image

And the directory where the provider is :
C:\Users\Rmuf003.terraform.d\plugins\registry.terraform.io\hashicorp\azurerm\3.44.0\windows_amd64

image
image793×110 4.65 KB