Ubuntu2404 login via ssh key

I’m trying to build Ubuntu 24.04 packer images using the ssh-key based login method, but I could not find any documents/links pointing on how to do it. I’m currently using the following block in packer json and the user-data block successively. But I see the error listed out. Appreciate any help.

      "ssh_private_key_file"    : "xxxx.pem",
      "ssh_port"                : 22,
      "ssh_wait_timeout"        : "60m",
      "ssh_username"            : "centos",

 users:
      - name: centos
        shell: /bin/bash
        ssh-authorized-keys:
          - "ssh-rsa <<key>> "
        sudo: ALL=(ALL) NOPASSWD:ALL
  keyboard:
    layout: us
  locale: en_US
  network:
    ethernets:
      eth0:
        dhcp4: true
        dhcp-identifier: mac
    version: 2
  ssh:
    allow-pw: true
    install-server: true
    allow-public-key-authentication: true
    authorized-keys:
      - "ssh-rsa <<key>>"
  storage:
[DEBUG] Detected authentication error. Increasing handshake attempts.
2025/06/24 21:22:58 packer-plugin-qemu_v1.1.2_x5.0_linux_amd64 plugin: 2025/06/24 21:22:58 failed to unlock port lockfile: close tcp 127.0.0.1:5900: use of closed network connection
==> qemu: Error waiting for SSH: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
2025/06/24 21:22:58 packer-plugin-qemu_v1.1.2_x5.0_linux_amd64 plugin: 2025/06/24 21:22:58 failed to unlock port lockfile: close tcp 127.0.0.1:4346: use of closed network connection

Thanks