TF 1.0.10: When removing count from module, plan wants to destroy then re-create everything

I started a TF project on v0.14.8 and at the time used count to select a module like so:

module "moduleA" { 
  source                         = "./moduleA"
  count                          = var.aws_env == "moduleA" ? 1 : 0
  aws_env                        = var.aws_env
  aws_security_group_id          = var.aws_security_group_id
  aws_subnet_1                   = var.aws_subnet_1
}

module "moduleB" {
  source                         = "./moduleB"
  count                          = var.aws_env == "moduleB" ? 1 : 0
  aws_env                        = var.aws_env
  aws_security_group_id          = var.aws_security_group_id
  aws_subnet_1                   = var.aws_subnet_1
}

When I would run terraform plan/apply, I would have the aws_env set to moduleA or moduleB to select that module.

I have since updated to v1.0.10 successfully and am looking to get rid of the count argument because of it’s inherent blockers with Providers within Modules. My latest main.tf looks like so:

terraform {
  required_version = ">= 1.0.10"
  backend "s3" {
  }
}

provider "aws" {
  region = "us-east-1"
}

module "moduleA" { 
  source                         = "./moduleA"  
  aws_env                        = var.aws_env
  aws_security_group_id          = var.aws_security_group_id
  aws_subnet_1                   = var.aws_subnet_1
}

module "moduleB" {
  source                         = "./moduleB"
  aws_env                        = var.aws_env
  aws_security_group_id          = var.aws_security_group_id
  aws_subnet_1                   = var.aws_subnet_1
}

variable "aws_env" {}
variable "aws_security_group_id" {}
variable "aws_subnet_1" {}

This works great conceptually as I plan/apply the TF like so ({TF_ACTION} is set to plan or apply):

terraform ${TF_ACTION} -target=module.${AWS_ENV} -input=false with {AWS_ENV} set to moduleA or moduleB.

The problem I’m facing is that once count has been removed, on a TF plan - TF wants to destroy and re-create all 155+ of my AWS resources - including KMS keys which are currently encrypting data.

How can I avoid TF wanting to destroy then re-create everything while removing count?