Terraform init error in windows Failed to query available provider packages

Terraform
#1 
Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/azurerm versions matching "2.46.0"...

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider
hashicorp/azurerm: could not connect to registry.terraform.io: Failed to
request discovery document: Get
"https://registry.terraform.io/.well-known/terraform.json": net/http: request
canceled while waiting for connection (Client.Timeout exceeded while awaiting
headers)

Able to browse following link : https://registry.terraform.io/.well-known/terraform.json

Terraform version : v0.13.5

#2

Hi @anand2803,

A common cause of errors like this is when using firewall software that imposes different blocking rules depending on which software is making a request. For example, a browser would typically already be approved to access remote servers, but the terraform.exe program might be blocked by default until you explicitly approve it.

Do you have any software configured on your system that might need to be configured to approve Terraform to access the network?

#3

Hi @apparentlymart,

Yes I do have couple of software like zscaler ,Crowdstrike,Forcepoint one endpoint, not sure how to check which one is culprit.
I enabled trace logging in terraform , and the output is

2021/04/08 10:16:13 [INFO] Go runtime version: go1.15.6
2021/04/08 10:16:13 [INFO] CLI args: string{“C:\Users\1912695\AppData\Local\Microsoft\WindowsApps\terraform.exe”, “init”}
2021/04/08 10:16:13 [DEBUG] Attempting to open CLI config file: C:\Users\1912695\AppData\Roaming\terraform.rc
2021/04/08 10:16:13 [DEBUG] File doesn’t exist, but doesn’t need to. Ignoring.
2021/04/08 10:16:13 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021/04/08 10:16:13 [DEBUG] ignoring non-existing provider search directory C:\Users\1912695\AppData\Roaming\terraform.d\plugins
2021/04/08 10:16:13 [DEBUG] ignoring non-existing provider search directory C:\Users\1912695\AppData\Roaming\HashiCorp\Terraform\plugins
2021/04/08 10:16:13 [INFO] CLI command args: string{“init”}

Init2021/04/08 10:16:13 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
ial2021/04/08 10:16:13 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
i2021/04/08 10:16:13 [DEBUG] New state was assigned lineage “c8fc78bd-9fd8-af7a-bc10-30187d3d6f8f”
zing the backend…2021/04/08 10:16:13 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2021/04/08 10:16:13 [TRACE] Meta.Backend: instantiated backend of type

2021/04/08 10:16:13 [DEBUG] checking for provisioner in “.”
2021/04/08 10:16:13 [DEBUG] checking for provisioner in “C:\Users\1912695\AppData\Local\Microsoft\WindowsApps”
2021/04/08 10:16:13 [INFO] Failed to read plugin lock file .terraform\plugins\windows_amd64\lock.json: open .terraform\plugins\windows_amd64\lock.json: The system cannot find the path specified.
2021/04/08 10:16:13 [TRACE] Meta.Backend: backend does not support operations, so wrapping it in a local backend
2021/04/08 10:16:13 [TRACE] backend/local: state manager for workspace “default” will:

  • read initial snapshot from terraform.tfstate
  • write new snapshots to terraform.tfstate
  • create any backup at terraform.tfstate.backup
    2021/04/08 10:16:13 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
    2021/04/08 10:16:13 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that’s okay
    2021/04/08 10:16:13 [TRACE] statemgr.Filesystem: read nil snapshot

2021/04/08 10:16:13 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
Initializing2021/04/08 10:16:13 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
provider plugins…

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider
hashicorp/azurerm: could not connect to registry.terraform.io: Failed to
request discovery document: Get
“https://registry.terraform.io/.well-known/terraform.json”: net/http: request
canceled while waiting for connection (Client.Timeout exceeded while awaiting
headers)

Regards

#4

I’m getting the same issue. This is inside WSL Ubuntu 20.04 on Windows. No strange firewall rules or setup (standard Windows 10 configuration). If I try to wget on any of the same URLs it succeeds, same for launching from browser. I have rebooted, restarted WSL Ubuntu. No change.

Oddly, I was upgrading a project from terraform 0.12 so I was running TF 0.13 to do the 0.13 upgrades and init and apply from 0.13 worked fine. But after I finished that and tried switching to 0.14, that’s when this init issue started. It seems like a big coincidence that there are network issues that started in the 3 minutes between my last 0.13 init and switching to 0.14?

LOG:

2021/04/08 13:49:29 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions
2021/04/08 13:49:29 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions
2021/04/08 13:49:39 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/08 13:49:39 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions: retrying in 1s (1 left)
2021/04/08 13:49:40 [INFO] Previous request to the remote registry failed, attempting retry.
2021/04/08 13:49:40 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions
2021/04/08 13:49:50 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/kubernetes/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/08 13:49:50 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/random/versions
2021/04/08 13:49:50 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/random/versions
- Finding latest version of hashicorp/random...
2021/04/08 13:50:00 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/random/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/random/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/08 13:50:00 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/random/versions: retrying in 1s (1 left)
2021/04/08 13:50:01 [INFO] Previous request to the remote registry failed, attempting retry.
2021/04/08 13:50:01 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/random/versions
2021/04/08 13:50:11 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/random/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/random/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/08 13:50:11 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/azuread/versions
- Finding hashicorp/azuread versions matching "1.4.0"...
2021/04/08 13:50:11 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/azuread/versions
2021/04/08 13:50:21 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/azuread/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/azuread/versions": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2021/04/08 13:50:21 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/azuread/versions: retrying in 1s (1 left)
2021/04/08 13:50:22 [INFO] Previous request to the remote registry failed, attempting retry.
2021/04/08 13:50:22 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/azuread/versions
2021/04/08 13:50:32 [ERR] GET https://registry.terraform.io/v1/providers/hashicorp/azuread/versions request failed: Get "https://registry.terraform.io/v1/providers/hashicorp/azuread/versions": dial tcp 192.33.14.30:443: i/o timeout
#5

Thanks for the note about the apparent change in behavior between v0.13 and v0.14, @jonstelly. That at least narrows down the space of possible explanations a little.

Reviewing the Terraform v0.14 changelog I see a few upgrade notes related to network/TLS requests that came as a result of us switching to a newer version of the Go runtime library:

  1. TLS certificate verification for outbound HTTPS requests from Terraform CLI no longer treats the certificate’s “common name” as a valid hostname when the certificate lacks any “subject alternative name” entries for the hostname. TLS server certificates must list their hostnames as a “DNS name” in the subject alternative names field.
  2. Outbound HTTPS requests from Terraform CLI now enforce RFC 8446’s client-side downgrade protection checks. This should not significantly affect normal operation, but may result in connection errors in environments where outgoing requests are forced through proxy servers and other “middleboxes”, if they have behavior that resembles a downgrade attack.
  3. Terraform’s HTTP client code is now slightly stricter than before in HTTP header parsing, but in ways that should not affect typical server implementations: Terraform now trims only ASCII whitespace characters, and does not allow Transfer-Encoding: identity.

All three of these seem like situations that ought to lead to an explicit parsing/validation error rather than a timeout, so it doesn’t seem likely to me that any of these changes would cause a regression like you’ve seen here, but I wanted to mention it in case it prompts an idea on your end, given that you have more context about how your computer and your network are configured than I do.

Since this error is emerging from the Go runtime library’s HTTP client, out of curiosity I went to see in what context it gets returned:

Unfortunately this seems not to really narrow anything down, because it seems like this send operation encapsulates everything from DNS lookup through the TLS handshake and then the actual HTTP transaction over the TLS channel, and so that doesn’t really give us a good clue as to what might be worth investigating deeper.

For WSL in particular though, I would typically also try using the native Windows executable outside of WSL to see if the behavior is different. I don’t have any particular expectation for it to be different, but any extra data helps when it comes to debugging.

#6

Thanks for the detailed response. I let things sit for a few hours and I just ran the same terraform init -upgrade and it ran fine this time. I’ll chalk it up to some sort of intermittent network issue somewhere along the line but if it happens again I’ll try and set up some network tracing.