Packer and vSphere: permissions issue

I am using Packer to programmatically build images as most here are. Packer is working with our two cloud provider environments without issue. I am having a permissions issue with Packer on our vSphere environment. I am trying to use an account that I am also using with Ansible Tower. I can stand up a vm from scratch on vSphere with an Ansible play with this account on Tower. I have tried my account as well which also does not work via Packer even though I can log-in to vSphere without error.

The typical error is:
Build ‘vsphere-iso’ errored after 7 seconds 26 milliseconds: ServerFaultCode: Permission to perform this operation was denied.

I have reviewed this article as well:

Packer version: 1.7.0
vSphere version: 6.7

The typical build syntax is the following with the creds being passed in:
$ PACKER_LOG=1 packer build -var-file vars.json linux-build.json