Custom AMI created by Packer not working with Systems Manager

So im currently learning how to codify infrastructure utilizing Packer and Terraform. I’ve been testing with AWS primarily for now. I created a packer image and was then using that image to deploy several windows instances with terraform. What i discovered was that after deploying with terraform, i could not connect to the instances via Systems Manager. After a good amount of troubleshooting, i discovered the packer image needs to be created in the same VPC/Subnet that Terraform will be using in order for AWS Systems Manager to work.

I had previously configured my packer script to deploy the ami in a different VPC/Subnet. But because the AMI’s appear to be visible across all VPC’s within the account, i didnt think it mattered where i originally deployed it from. So there appears to be some networking dependencies that gets put into the template. Is there a way to wipe that out so that i can utilize an AMI from any VPC? or is that just the way the product is designed, and i will need to re-think my terraform scripts to accommodate any custom AMI’s i might want to use.

i originally had terraform deploying a vpc, subnets, security group, internet gateway, route, and instances. But if the infrastructure needs to exist prior for packer, i might need to remove the instance portion.