Scalable targets are not destroyed when destroying EMR clusters having instance groups with autoscaling policies

Terraform Providers AWS
1

We are creating multiple clusters on demand for testing purposes. After testing is done, the clusters are destroyed. We may create and destroy around 15-20 clusters per day.

Recently we noticed that on new EMR clusters, autoscaling policies fail to attach with reason “Failed to provision the AutoScaling policy: The maximum limit of 500 scalable targets for service namespace elasticmapreduce has been reached”.

Listing scalable targets with “aws application-autoscaling describe-scalable-targets --service-namespace elasticmapreduce” shows 500 registered scalable targets belonging to old, non-existing EMR clusters.

Is this expected behavior? Should we manually deregister scalable targets after destroying EMR resources?

To reproduce:

  1. Create an EMR cluster with an autoscaling policy attached to an instance group
  2. Destroy created resources
  3. Run “aws application-autoscaling describe-scalable-targets --service-namespace elasticmapreduce”, this will show a registered scalable target from the destroyed EMR cluster
2

I got the same problem! Can not figure out how to deregister these auto scaling policies from terraform.
Please let me know If you got the solution.