Packer v1.10.1 Ansible Provisioner ssh issue

We have a packer build that calls an ansible provisioner, but 2 out of 3 engineers keep failing on ansible ssh’ing to new vm it is creating.

It is failing on this step where ansible is trying to move a tmp file it needs from the engineers local to the new vm.

vsphere-clone.linux-ubuntu: fatal: [default]: FAILED! => {“msg”: “failed to transfer file to /Users/ap428/.ansible/tmp/ansible-local-85466jpdr2g2n/tmp2uooe8cf /home//.ansible/tmp/ansible-tmp-1713966824.0018728-85474-206464356620374/\n\n”}
vsphere-clone.linux-ubuntu: PLAY RECAP *********************************************************************
vsphere-clone.linux-ubuntu: default : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
==> vsphere-clone.linux-ubuntu: Error executing Ansible: Non-zero exit status: exit status 2

The only thing different between the 3 that we can find is the Mac OS. The one that works is using Monterey. The two that fail, one is on Ventura and the other is on Sonoma. Has anyone else seen this or have any ideas of how to get around it? We have tried everything we have found on google so far and nothing has worked.

Found it was trying to go to localhost so added in use_proxy to false.

vsphere-clone.linux-ubuntu: <> (0, b’‘, b’')
vsphere-clone.linux-ubuntu: fatal: [default]: FAILED! => {
vsphere-clone.linux-ubuntu: “msg”: “failed to transfer file to /Users/ap428/.ansible/tmp/ansible-local-3712363wibbxc/tmpqy09976j /home//.ansible/tmp/ansible-tmp-1715114623.1960042-37134-124545107511591/\n\n”

Now it is trying to use the VM’s IP address, but get the following error.

vsphere-clone.linux-ubuntu: <> (255, b’‘, b’Warning: Permanently added '' (ED25519) to the list of known hosts.\r\nLoad key “/var/folders/zw/dzlynk910b13jbl44tth0cy40000gp/T/ansible-key680473184”: invalid format\r\n@ Permission denied (publickey,password).\r\n’)
vsphere-clone.linux-ubuntu: fatal: [default]: UNREACHABLE! => {
vsphere-clone.linux-ubuntu: “changed”: false,

This is my ansible provisioner in packer.
build {
sources = [“source.vsphere-clone.linux-ubuntu”]

provisioner “ansible” {
user = var.build_username
galaxy_file = “{path.cwd}/ansible/requirements.yaml" galaxy_force_with_deps = true playbook_file = "{path.cwd}/ansible/ubuntu-20-04-hardened.yaml”
roles_path = “${path.cwd}/ansible/roles”
use_proxy = false
extra_arguments = [
“–tags”, “build, section1, section2, section3, section4, section5, section6”

This is my ansbile.cfg file

stdout_callback = default
display_skipped_hosts = false
ansible_python_interpreter = /opt/homebrew/bin/python3
verbosity = 4
host_key_checking = False

ssh_args = -o ControlMaster=no -o ControlPersist=no -o ControlPath=none -o UserKnownHostsFile=/dev/null LogLevel=DEBUG3 -o ServerAliveInterval=60 -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedKeyTypes=ssh-rsa -o ForwardAgent=yes
pipelining = false
scp_extra_args = -O
scp_if_ssh = true
transfer_method = smart

OpenSSH_9.6p1, LibreSSL 3.3.6
Python 3.12.3
Ansible 9.5.1