Terraform crash for aws project

prodipto03ei38 · March 13, 2023, 3:12pm

Terraform v0.13.0
The project provision was 8 months before. just updating the tags and got the error.
Terraform crash

Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Canceled desc = context canceled
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = transport is closing
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
Error: rpc error: code = Unavailable desc = connection error: desc = “transport: error while dialing: dial unix /tmp/plugin577458583: connect: connection refused”
panic: Error reading level state: strconv.ParseInt: parsing “42757242880”: value out of range
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5:
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: goroutine 408 [running]:
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).get(0x1fc9cfa0, {0x1fcd2300, 0x1, 0x1}, 0x8)
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2@v2.25.0/helper/schema/resource_data.go:553 +0x279
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).State(0x1fc9cfa0)
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2@v2.25.0/helper/schema/resource_data.go:351 +0x3a7
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).ShimInstanceStateFromValue(0x1c939490, {{{0x147b3b40, 0x1fcb5718}}, {0x119b5dc0, 0x1e4c1d20}})
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2@v2.25.0/helper/schema/resource.go:599 +0x1d9
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0x1df88990, {0x147b32c0, 0x1fcc45a0}, 0x1e4c1580)
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-sdk/v2@v2.25.0/helper/schema/grpc_provider.go:5 87 +0x23f
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0x1df9e360, 0x1df9e3a0, {0x1e8fdad0, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, …}, …}, …)
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-mux@v0.9.0/tf5muxserver/mux_server_ReadResource.go:26 +0x151
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0x1ec1b130, {0x147b32c0, 0x1fcc4228}, 0x1fc927b0)
2023-03-13T14:36:24.559Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/tf5server/server.go:748 +0x4c5
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x131f3620, 0x1ec1b130}, {0x147b32c0, 0x1fcc4228}, 0x1fc84740, 0x0)
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: github.com/hashicorp/terraform-plugin-go@v0.14.3/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x18a
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc.(*Server).processUnaryRPC(0x1cbd9680, {0x147b8c10, 0x1c526360}, 0x1f8fa820, 0x1e176990, 0x1a9d82c8, 0x0)
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc@v1.53.0/server.go:1336 +0xe22
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc.(*Server).handleStream(0x1cbd9680, {0x147b8c10, 0x1c526360}, 0x1f8fa820, 0x0)
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc@v1.53.0/server.go:1704 +0xa17
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc.(*Server).serveStreams.func1.2()
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc@v1.53.0/server.go:965 +0xb1
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: created by google.golang.org/grpc.(*Server).serveStreams.func1
2023-03-13T14:36:24.560Z [DEBUG] plugin.terraform-provider-aws_v4.58.0_x5: google.golang.org/grpc@v1.53.0/server.go:963 +0x25f
2023/03/13 14:36:24 [ERROR] eval: *terraform.EvalRefresh, err: rpc error: code = Unavailable desc = transport is closing
2023/03/13 14:36:24 [ERROR] eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2023/03/13 14:36:24 [TRACE] [walkRefresh] Exiting eval tree:

!!! TERRAFORM CRASH !!!
Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at “crash.log” relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.
When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running ‘terraform --version’ on the command line.
SECURITY WARNING: the “crash.log” file that was created may contain
sensitive information that must be redacted before it is safe to share
on the issue tracker.
[1]: Issues · hashicorp/terraform · GitHub
!!! TERRAFORM CRASH !!!
Cleaning up file based variables
00:00
ERROR: Job failed: exit status 1

maxb · March 13, 2023, 4:23pm

I think it’s suggesting that your Terraform state file contains the value 42757242880 from a previous apply, and something has changed in terraform-provider-aws (I notice you’re running a version published 4 days ago, so must have updated it since 8 months ago) and the new version of the provider is crashing when trying to read that data.

I’d start by trying to roll back the version of terraform-provider-aws to whatever you used before, whilst also having a look in the state to see in what context 42757242880 appears.

jbardin · March 13, 2023, 5:40pm

@prodipto03ei38, I can’t recall if any changes in core were mode to help work around large integer problems like this, but that is a very old version of Terraform which could be involved. If you can’t upgrade soon, at the very least you should be running the latest patch release, which was v0.13.7.

prodipto03ei38 · March 14, 2023, 9:15am

This is the value of elasticfilesystem in bytes, I checked in the tfstate file.
{
“value”: 42757242880,
“value_in_ia”: 0,
“value_in_standard”: 42757242880
}