Hi There.
terraform version :1.5.4
aws provider version: 4.67.0
We run a number of EMR clusters to handle Spark structured streaming jobs. In nonprod environments we deploy clusters daily with a set number of core nodes then create a number of task instance groups of varying instance types that can be called on to fill capacity as required. Task fleet config is unable to be used in our scenario.
We have these instance groups set with instance count as zero, instance type and bid price.
The provider creates these instance groups on cluster creation, but it creates the group and then spins up an instance of that type, then immediately resizes back down to zero.
Is this expected behaviour?
Is there a way to prevent it?
As I see it this slows down the terraform deployment pipeline a fair bit, waiting for the instance deployment and subsequent destruction, and incurs unnecessary costs.
I have spoken with AWS support and they can see that the cluster is performing what is being requested via the API, so the behaviour is being controlled within the aws provider code.