Introduction
Hi guys, I am trying to get started with contributing to the terraform-provider-azurerm. I have noticed a problem with the azurerm_firewall_network_rule_collection I have reported it here. I thought I should take this as an opportunity to see if I can chip in and try to fix the problem I have reported! If I can debug the provider and see how the code works, then not only will I learn go, I will also be able to figure out and help with bug fixes/feature enhancements.
Any Help you guys can provide me and guide me towards the right direction will be much appreciated. I know your time is precious, and I did my best before coming to ask for help.
Summary of what I have tried to do the folllowing:
-
Fork the
terraform-provider-azurermrepository. -
Clone the forked repo to my local. I am using
Ubuntu 20.04in WSL. -
Running
vscodeversion1.54.3 -
Install
golangand related go tools in vscode. I am ongo version go1.16.2 linux/amd64. Usedlinuxbrewto install go. -
Install
gcc5. Usedlinuxbrewto install. -
Install
delvedebugger. Usedlinuxbrewto install. -
Create a very simple terraform
main.tfwhich just deploys a resource group to my Azure subscription. -
Create a
.terraformrcfile ( set it in the location/home/rahul/.terraformrc) to supply dev_overrides path to the provider. -
Modify the
main.go( location/usr/local/go/src/github.com/terraform-providers/terraform-provider-azurerm/main.go) to use debug statements. -
Compile the provider with the debug code and create new binaries.
-
Attach dlv in headless mode to this binary
-
Use the data from the previous step to set it in the variable TF_REATTACH_PROVIDERS
-
Run Terraform plan - hoping that have a successful plan runs.
-
Turn off debug mode and compile the provider and then run terraform plan.
-
Plan runs successfully using the local provider.
The Problem
-
The plan crashes with
Terraform crashed! This is always indicative of a bug within Terraform. A crash log has been placed at "crash.log" relative to your current working directory. It would be immensely helpful if you could please report the crash with Terraform[1] so that we can fix this. -
Content from log output
2021/04/05 06:09:50 [INFO] Terraform version: 0.14.9
2021/04/05 06:09:50 [INFO] Go runtime version: go1.15.6
2021/04/05 06:09:50 [INFO] CLI args: []string{"/home/rahul/.tfenv/versions/0.14.9/terraform", "plan"}
2021/04/05 06:09:50 [DEBUG] Attempting to open CLI config file: /home/rahul/.terraformrc
2021/04/05 06:09:50 Loading CLI configuration from /home/rahul/.terraformrc
2021/04/05 06:09:50 [DEBUG] Explicit provider installation configuration is set
2021/04/05 06:09:50 [TRACE] Selected provider installation method cliconfig.ProviderInstallationDirect with includes [] and excludes []
2021/04/05 06:09:50 [INFO] CLI command args: []string{"plan"}
2021/04/05 06:09:50 [TRACE] Meta.Backend: built configuration for "local" backend with hash value 666019178
2021/04/05 06:09:50 [TRACE] Preserving existing state lineage "8773571c-7605-2553-b418-08cbb37374a4"
2021/04/05 06:09:50 [TRACE] Preserving existing state lineage "8773571c-7605-2553-b418-08cbb37374a4"
2021/04/05 06:09:50 [TRACE] Meta.Backend: working directory was previously initialized for "local" backend
2021/04/05 06:09:50 [TRACE] Meta.Backend: using already-initialized, unchanged "local" backend configuration
2021/04/05 06:09:50 [TRACE] Meta.Backend: instantiated backend of type *local.Local
2021/04/05 06:09:50 [TRACE] providercache.fillMetaCache: scanning directory .terraform/providers
2021/04/05 06:09:50 [TRACE] getproviders.SearchLocalDirectory: .terraform/providers is a symlink to .terraform/providers
2021/04/05 06:09:50 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/azurerm v2.53.0 for linux_amd64 at .terraform/providers/registry.terraform.io/hashicorp/azurerm/2.53.0/linux_amd64
2021/04/05 06:09:50 [TRACE] providercache.fillMetaCache: including .terraform/providers/registry.terraform.io/hashicorp/ azurerm/2.53.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/azurerm 2.53.0
2021/04/05 06:09:50 [DEBUG] Provider registry.terraform.io/hashicorp/azurerm is overridden to load from /usr/local/go/bin
2021/04/05 06:09:50 [DEBUG] checking for provisioner in "."
2021/04/05 06:09:50 [DEBUG] checking for provisioner in "/home/rahul/.tfenv/versions/0.14.9"
2021/04/05 06:09:50 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/ plugins/linux_amd64/lock.json: no such file or directory
2021/04/05 06:09:50 [TRACE] Meta.Backend: backend *local.Local supports operations
2021/04/05 06:09:50 [INFO] backend/local: starting Plan operation
2021/04/05 06:09:50 [TRACE] backend/local: requesting state manager for workspace "default"
2021/04/05 06:09:50 [TRACE] backend/local: state manager for workspace "default" will:
- read initial snapshot from terraform.tfstate
- write new snapshots to terraform.tfstate
- create any backup at terraform.tfstate.backup
2021/04/05 06:09:50 [TRACE] backend/local: requesting state lock for workspace "default"
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: preparing to manage state snapshots at terraform.tfstate
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: no previously-stored snapshot exists
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: locking terraform.tfstate using fcntl flock
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: writing lock metadata to .terraform.tfstate.lock.info
2021/04/05 06:09:50 [TRACE] backend/local: reading remote state for workspace "default"
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: reading latest snapshot from terraform.tfstate
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2021/04/05 06:09:50 [TRACE] statemgr.Filesystem: read nil snapshot
2021/04/05 06:09:50 [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2021/04/05 06:09:50 [TRACE] backend/local: building context for current working directory
2021/04/05 06:09:50 [TRACE] terraform.NewContext: starting
2021/04/05 06:09:50 [TRACE] terraform.NewContext: loading provider schemas
2021/04/05 06:09:50 [TRACE] LoadSchemas: retrieving schema for provider type "registry.terraform.io/hashicorp/azurerm"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x20f82d2]
goroutine 52 [running]:
github.com/hashicorp/terraform/plugin.(*GRPCProviderPlugin).Client(0xc00054ef60, 0xc00054e560, 0xc0003321e0, 0x7f73af739388, 0xc00094a180, 0x0, 0xc000817a6c)
<autogenerated>:1 +0x32
github.com/hashicorp/go-plugin.(*RPCClient).Dispense(0xc00073ce40, 0x289bfc5, 0x8, 0x0, 0x0, 0x29cd9d8, 0xc0003a6b30)
/go/pkg/mod/github.com/hashicorp/go-plugin@v1.3.0/rpc_client.go:159 +0x254
github.com/hashicorp/terraform/command.unmanagedProviderFactory.func1(0x24b0360, 0xc000575950, 0xc0003a6c08, 0xc00000b270)
/home/circleci/project/project/command/meta_providers.go:448 +0x248
github.com/hashicorp/terraform/terraform.(*basicComponentFactory).ResourceProvider(0xc00017d4b0, 0xc000578d8a, 0x7, 0xc000578d80, 0x9, 0x28bc2ba, 0x15, 0x2892906, 0x1, 0xc000578d8a, ...)
/home/circleci/project/project/terraform/context_components.go:55 +0x351
github.com/hashicorp/terraform/terraform.loadProviderSchemas.func1(0xc000578d8a, 0x7, 0xc000578d80, 0x9, 0x28bc2ba, 0x15)
/home/circleci/project/project/terraform/schemas.go:103 +0x36e
github.com/hashicorp/terraform/terraform.loadProviderSchemas(0xc000731ec0, 0xc0002c11e0, 0x0, 0x2ce93a0, 0xc00017d4b0, 0xc0000f2080, 0x3dcafe0, 0xc0000f2050)
/home/circleci/project/project/terraform/schemas.go:174 +0x250
github.com/hashicorp/terraform/terraform.LoadSchemas(0xc0002c11e0, 0x0, 0x2ce93a0, 0xc00017d4b0, 0x0, 0x2, 0x7f73afb6b400)
/home/circleci/project/project/terraform/schemas.go:84 +0xc6
github.com/hashicorp/terraform/terraform.NewContext(0xc0003a79d0, 0x0, 0x0, 0xc00098b900, 0x1)
/home/circleci/project/project/terraform/context.go:205 +0x2ea
github.com/hashicorp/terraform/backend/local.(*Local).contextDirect(0xc00015ef70, 0xc0008e2700, 0xc0002c11e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc000731d10, 0xc000421120, ...)
/home/circleci/project/project/backend/local/backend_local.go:178 +0x2ff
github.com/hashicorp/terraform/backend/local.(*Local).context(0xc00015ef70, 0xc0008e2700, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/circleci/project/project/backend/local/backend_local.go:111 +0xf18
github.com/hashicorp/terraform/backend/local.(*Local).opPlan(0xc00015ef70, 0x2ce83a0, 0xc00073cb40, 0x2ce83a0, 0xc00073cb80, 0xc0008e2700, 0xc00073cb00)
/home/circleci/project/project/backend/local/backend_plan.go:70 +0x2cb
github.com/hashicorp/terraform/backend/local.(*Local).Operation.func1(0xc00017d130, 0xc00017d140, 0xc00017d150, 0xc00015ef70, 0xc00017d120, 0x2ce83a0, 0xc00073cb40, 0x2ce83a0, 0xc00073cb80, 0xc0008e2700, ...)
/home/circleci/project/project/backend/local/backend.go:340 +0xe8
created by github.com/hashicorp/terraform/backend/local.(*Local).Operation
/home/circleci/project/project/backend/local/backend.go:334 +0x36d
Detailed steps including source code.
I have followed the steps documented in Terraform Docs - Debugger based debugging
-
Go Installation path =>
/usr/local/go -
terraform-provider-azurerm cloned to =>
/usr/local/go/src/github.com/terraform-providers -
terraform-provider-azurerm source path =>
/usr/local/go/src/github.com/terraform-providers/terraform-provider-azurerm -
Terraform main.tf source path =>
~/Code/terraform -
My terraform code
locals { location = "australiaeast" resourcegroup = "rg-azfw" } terraform { backend "local" {} } terraform { required_providers { azurerm = { source = "hashicorp/azurerm" version = "2.53.0" } } } provider "azurerm" { tenant_id = "xxxxxx-xxxxx-xxxxx-xxxxx-xxxx" skip_provider_registration = true features {} } resource "azurerm_resource_group" "resourcegroup" { name = local.resourcegroup lo -
My .terraformrc
provider_installation { # Use /home/developer/go/bin as an overridden package directory # for the hashicorp/azurerm provider. This disables the version and checksum # verifications for this provider and forces Terraform to look for the # azurerm provider plugin in the given directory. dev_overrides { "registry.terraform.io/hashicorp/azurerm" = "/usr/local/go/bin" } # For all other providers, install them directly from their origin provider # registries as normal. If you omit this, Terraform will _only_ use # the dev_overrides block, and so no other providers will be available. direct {} } -
Modified the main.go file in
terraform-provider-azurermso that the provider binary can be launched in debugger mode. Now the main function looks like :func main() { // remove date and time stamp from log output as the plugin SDK already adds its own log.SetFlags(log.Flags() &^ (log.Ldate | log.Ltime)) var debugMode bool flag.BoolVar(&debugMode, "debug", true, "set to true to run the provider with support for debuggers like delve") flag.Parse() opts := &plugin.ServeOpts{ProviderFunc: azurerm.Provider} if debugMode { // TODO: update this string with the full name of your provider as used in your configs err := plugin.Debug(context.Background(), "registry.terraform.io/hashicorp/azurerm", opts) if err != nil { log.Fatal(err.Error()) } return } plugin.Serve(opts) // plugin.Serve(&plugin.ServeOpts{ // ProviderFunc: azurerm.Provider, // }) } -
The vscode debugger launch config looks like :
{ // Use IntelliSense to learn about possible attributes. // Hover to view descriptions of existing attributes. // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "name": "Connect to server", "type": "go", "request": "attach", "mode": "remote", "remotePath": "${workspaceFolder}", "port": 36283, "host": "127.0.0.1", "apiVersion": 1 } ] } -
Ran
make buildto build the binaries. It copies the binary to$GOPATH/bin/location. -
Launching the delv headless debugger with the command
dlv exec --listen=127.0.0.1:36283 --api-version=1 --headless $GOPATH/bin/terraform-provider-azurerm -- --debug- The
--listen=127.0.0.1:36283is an additional parameter I googled and figured out - not mentioned in terraform docs. This helps me in not changing the ports for debugging everytime I am launching the debugger.
- The
-
That produces an output like :
API server listening at: 127.0.0.1:36283 {"@level":"debug","@message":"plugin address","@timestamp":"2021-04-05T06:08:38.863169+10:00","address":"/tmp/ plugin594405865","network":"unix"} Provider server started; to attach Terraform, set TF_REATTACH_PROVIDERS to the following: {"registry.terraform.io/hashicorp/azurerm":{"Protocol":"netrpc","Pid":16362,"Test":true,"Addr":{"Network":"unix","String":"/ tmp/plugin594405865"}}} -
Set the
TF_REATTACH_PROVIDERSvalueexport TF_REATTACH_PROVIDERS='{"registry.terraform.io/hashicorp/azurerm":{"Protocol":"netrpc","Pid":16362,"Test":true,"Addr":{"Network":"unix","String":"/tmp/plugin594405865"}}}' -
Run the
terraform planin the folder where the tf source code is there… And all goes boom.
(I expect other providers are working in spite of this because they’ve dropped Terraform v0.11 support already and so they treat protocol 5 and grpc as the baseline.)
Thanks for taking out the time to go through it.