Terraform hangs on aws_api_gateway_authorizer creation

Terraform Providers AWS
1

I am trying to create an api gateway authorizer on AWS but terraform hangs on creation with the following error “Unable to complete operation due to concurrent modification. Please try again later.”

All other resources are created just fine, it is only the authorizer that it gets stuck on.

Things I tried:

  • Waited for at least 40 mins for the authorizer to get created
  • Upgraded tf to the latest version - I am on 1.6.1
  • Upgrade aws provider to the latest version - 5.21.0
  • Run terraform apply with parallelism=1
  • Create just the authorizer resource

I am out of ideas how to solve this. Can anyone please help?

My configuration:

resource "aws_apigatewayv2_api" "demo-api-gw" {
  name                          = "${local.product}-${local.demo_name}-${local.demo_num}-api-gw"
  protocol_type                 = "HTTP"

  cors_configuration  {   
    allow_credentials           = false
    allow_headers               = ["*"]
    allow_methods               = ["GET", "OPTIONS", "POST"]
    allow_origins               = ["*"]
    expose_headers              = ["*"]
    max_age                     = 60
  }
}

resource "aws_iam_role" "demo-authorizers-invocation-role" {
    name                    = "${local.product}-${local.demo_name}-${local.demo_num}-authorizers-invocation-role"
    path                    = "/"
    assume_role_policy      = data.aws_iam_policy_document.demo-api-gateway-sts-assume-role-policy-doc.json
}

resource "aws_api_gateway_authorizer" "demo-authorizers" {
  name                   = "${local.product}-${local.demo_name}-${local.demo_num}-devices-authorizer"
  rest_api_id            = aws_apigatewayv2_api.demo-api-gw.id
  authorizer_uri         = aws_lambda_function.demo-authorizers-lambda.invoke_arn
  type                   = "REQUEST"
  authorizer_credentials = aws_iam_role.demo-authorizers-invocation-role.arn
  authorizer_result_ttl_in_seconds = 1
}


data "aws_iam_policy_document" "demo-api-gateway-sts-assume-role-policy-doc" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["apigateway.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

DEBUG OUTPUT:

 2023-10-16T13:00:25.284+0300 [DEBUG] provider.terraform-provider-aws_v5.21.0_x5.exe: HTTP Request Sent: http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************JM4A/20231016/us-east-1/apigateway/aws4_request, SignedHeaders=accept;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" http.request.header.content_type=application/json http.request.header.x_amz_security_token="*****" net.peer.name=apigateway.us-east-1.amazonaws.com http.request.header.x_amz_date=20231016T100025Z tf_aws.sdk=aws-sdk-go
  http.request.body=
  | {"authorizerCredentials":"arn:aws:iam::XXXXXXXXXXXX:role/d2-authorizers-invocation-role","authorizerResultTtlInSeconds":1,"authorizerUri":"arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:XXXXXXXXXXXX:function:d2-authorizers/invocations","identitySource":"method.request.header.Authorization","name":"bmc-hedge-roboticarms-d2-devices-authorizer","type":"REQUEST"}
   http.request.header.accept=application/json @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.38/logger.go:109 http.request_content_length=445 rpc.system=aws-api tf_resource_type=aws_api_gateway_authorizer aws.region=us-east-1 http.url=https://apigateway.us-east-1.amazonaws.com/restapis/vmyreqq3bh/authorizers http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.6.1 (+https://www.terraform.io) terraform-provider-aws/5.21.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.45.24 (go1.20.8; windows; 386)" tf_req_id=ede180d7-937c-1bc5-b66d-9bd719568bd4 tf_rpc=ApplyResourceChange @module=aws http.flavor=1.1 http.method=POST rpc.method=CreateAuthorizer rpc.service="API Gateway" tf_mux_provider="*schema.GRPCProviderServer" tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp="2023-10-16T13:00:25.284+0300"
2023-10-16T13:00:25.490+0300 [DEBUG] provider.terraform-provider-aws_v5.21.0_x5.exe: HTTP Response Received: @module=aws aws.region=us-east-1 http.response.header.x_amz_apigw_id=M451AITuIAMEclw= http.response.header.x_amzn_requestid=1a22cdea-ee5c-4eed-b41c-0871c12a7e34 tf_req_id=ede180d7-937c-1bc5-b66d-9bd719568bd4 tf_resource_type=aws_api_gateway_authorizer http.duration=205 rpc.method=CreateAuthorizer tf_aws.sdk=aws-sdk-go http.response.header.content_type=application/json http.response.header.date="Mon, 16 Oct 2023 10:00:25 GMT" http.response.header.x_amzn_errortype=ConflictException http.response_content_length=99 @caller=github.com/hashicorp/aws-sdk-go-base/v2/awsv1shim/v2@v2.0.0-beta.38/logger.go:157
  http.response.body=
  | {"message":"Unable to complete operation due to concurrent modification. Please try again later."}
   http.status_code=409 tf_rpc=ApplyResourceChange rpc.service="API Gateway" rpc.system=aws-api tf_mux_provider="*schema.GRPCProviderServer" tf_provider_addr=registry.terraform.io/hashicorp/aws timestamp="2023-10-16T13:00:25.490+0300"