Packer stuck at ssh - with Ubuntu Bionic ISO

I’m trying to build a qcow2 from custom ubuntu 18.04.2 ISO.
The VM boots up fine, but packer is not able to ssh. Halts with SSH handshake error.
When I connect to the VM via VNC and check, SSH is up and running, also
locally if I try to login via the same user - it works!
Looks like the TCP forward from Qemu isn’t working (guess), because when I
run journalctl -u ssh I dont see any messages of SSH being attempted!
Tried to restart ssh service, tried rebooting the VM but nothing helped.

Manually if I try ssh too it’s stuck for a long time and erros out:

ssh -vvv -p 2225 admin@
debug1: identity file /home/ani/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4

ssh_exchange_identification: Connection closed by remote host

I am using packer version v1.5.4.

Any tips on how to debug further ?