Ansible provisioner

I would like to use the ansible (not local) provisioner with a qemu builder.
I see just about every question results in someone advising to use the ansible-local which means installing ansible into the image first. This is really counter to what I like about ansible, which is NO agent needed.

I followed the packer docs for the ansible provisioner and set up a json. When the image is done, and reboots, ansible errors out because it is waiting for a sudo password.

There doesnt seem to be a way for me to tell the ansible provisioner this, and the docs seem to suggest a temporary ssh key is created for doing this anyway.

I’m sure I am missing something about how the provisioner works, but cannot figure out what it is, whether it is a Debian thing or a qemu thing or?