SSH not connecting to packer VM QEMU KVM - Waiting for SSH to become available

Hi

I am using qemu builder to build and customize Rocky Linux cloud images with cloud-init. and I am getting SSH timeouts. I also using ephemeral SSH keys and dynamically generating the user-data file.

Any ideas to get this working please?

Here is the output:

qemu.rocky9-cloud: output will be in this color.

2025-04-17T22:24:48+01:00: ==> qemu.rocky9-cloud: Retrieving ISO
2025-04-17T22:24:48+01:00: ==> qemu.rocky9-cloud: Trying file:///home/unix/gitlab/solaris9000/proxmox-packer-qemu/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2
2025-04-17T22:24:48+01:00: ==> qemu.rocky9-cloud: Trying file:///home/unix/gitlab/solaris9000/proxmox-packer-qemu/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2?checksum=sha256%3Af1cc39f7ee75974243a643466da5286cc20d3f4da3b159b3a147fcfc10fd4cf3
2025-04-17T22:24:50+01:00: ==> qemu.rocky9-cloud: file:///home/unix/gitlab/solaris9000/proxmox-packer-qemu/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2?checksum=sha256%3Af1cc39f7ee75974243a643466da5286cc20d3f4da3b159b3a147fcfc10fd4cf3 => /home/unix/gitlab/solaris9000/proxmox-packer-qemu/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2
2025-04-17T22:24:50+01:00: ==> qemu.rocky9-cloud: Creating CD disk...
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Drive current: -outdev 'stdio:/tmp/packer1329681071.iso'
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Media current: stdio file, overwriteable
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Media status : is blank
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Media summary: 0 sessions, 0 data blocks, 0 data, 28.1g free
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Added to ISO image: directory '/'='/tmp/packer_to_cdrom1373089530'
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: xorriso : UPDATE :       2 files added in 1 seconds
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: xorriso : UPDATE :       2 files added in 1 seconds
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: ISO image produced: 185 sectors
2025-04-17T22:24:50+01:00:     qemu.rocky9-cloud: Written to medium : 185 sectors at LBA 0
2025-04-17T22:24:51+01:00:     qemu.rocky9-cloud: Writing to 'stdio:/tmp/packer1329681071.iso' completed successfully.
2025-04-17T22:24:51+01:00:     qemu.rocky9-cloud: Done copying paths from CD_dirs
2025-04-17T22:24:51+01:00:     qemu.rocky9-cloud: File extension already matches desired output format. Skipping qemu-img convert step
2025-04-17T22:24:51+01:00: ==> qemu.rocky9-cloud: Resizing hard drive...
2025-04-17T22:24:51+01:00: ==> qemu.rocky9-cloud: Found port for communicator (SSH, WinRM, etc): 2504.
2025-04-17T22:24:51+01:00: ==> qemu.rocky9-cloud: Creating temporary RSA SSH key for instance...
2025-04-17T22:24:52+01:00: ==> qemu.rocky9-cloud: Looking for available port between 5960 and 5960 on 0.0.0.0
2025-04-17T22:24:52+01:00: ==> qemu.rocky9-cloud: Starting VM, booting disk image
2025-04-17T22:24:52+01:00:     qemu.rocky9-cloud: The VM will be run headless, without a GUI. If you want to
    qemu.rocky9-cloud: view the screen of the VM, connect via VNC without a password to
    qemu.rocky9-cloud: vnc://0.0.0.0:5960
2025-04-17T22:24:54+01:00: ==> qemu.rocky9-cloud: Waiting 10s for boot...
2025-04-17T22:25:05+01:00: ==> qemu.rocky9-cloud: Connecting to VM via VNC (0.0.0.0:5960)
2025-04-17T22:25:05+01:00: ==> qemu.rocky9-cloud: Typing the boot commands over VNC...
2025-04-17T22:25:05+01:00:     qemu.rocky9-cloud: Not using a NetBridge -- skipping StepWaitGuestAddress
2025-04-17T22:25:05+01:00: ==> qemu.rocky9-cloud: Using SSH communicator to connect: 127.0.0.1
2025-04-17T22:25:05+01:00: ==> qemu.rocky9-cloud: Waiting for SSH to become available...
2025-04-17T22:30:05+01:00: ==> qemu.rocky9-cloud: Timeout waiting for SSH.
2025-04-17T22:30:05+01:00: ==> qemu.rocky9-cloud: Deleting output directory...
2025-04-17T22:30:05+01:00: Build 'qemu.rocky9-cloud' errored after 5 minutes 16 seconds: Timeout waiting for SSH.

==> Wait completed after 5 minutes 16 seconds

==> Some builds didn't complete successfully and had errors:
--> qemu.rocky9-cloud: Timeout waiting for SSH.

==> Builds finished but no artifacts were created.

Here is the template

packer {
  required_version = ">= 1.11.2"
  required_plugins {
    qemu = {
      version = ">= 1.0.9"
      source  = "github.com/hashicorp/qemu"
    }
  }
}

locals {
  timestamp = formatdate("YYYY-MM-DD-HHmmss", timestamp())
}

source "qemu" "rocky9-cloud" {
  # Use Rocky 9 GenericCloud image as the base
  disk_image       = true
  iso_url          = "file://${path.cwd}/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2"
  iso_checksum     = "file:${path.cwd}/isos/Rocky-9-GenericCloud-LVM-9.5-20241118.0.x86_64.qcow2.CHECKSUM"
  iso_target_path  = "isos/Rocky-9-GenericCloud.qcow2"

  # QEMU settings
  qemu_binary      = "/usr/libexec/qemu-kvm"
  accelerator      = "kvm"
  disk_interface   = "virtio-scsi"
  format           = "qcow2"
  net_device       = "virtio-net"
  disk_compression = false

  # VNC configuration
  vnc_bind_address = "0.0.0.0"
  vnc_port_min = 5960
  vnc_port_max = 5960

  # System resources
  cpus      = 2
  memory    = 2048
  disk_size = "20G"

  # Output
  output_directory = "output-${local.timestamp}"
  vm_name          = "rocky9-cloud-${local.timestamp}.qcow2"

  # Cloud-init configuration
  cd_files = [
    "./cloud-init/meta-data",
    "./cloud-init/user-data"
  ]
  cd_label = "cidata"

  # SSH configuration - let Packer generate an ephemeral key
  ssh_username = "rocky"
  ssh_timeout  = "5m"
  # No ssh_private_key_file means Packer will generate an ephemeral key

  # Boot and shutdown
  boot_wait        = "10s"
  shutdown_command = "sudo shutdown -P now"
  shutdown_timeout = "2m"

  # Headless mode
  headless = true
}

build {
  sources = ["source.qemu.rocky9-cloud"]

  # Add log message about key generation
  provisioner "shell-local" {
    inline = [
      "echo '[Security] Packer has generated an ephemeral SSH key for this build'",
      "echo '[Security] This key will be automatically deleted when the build completes'"
    ]
  }

  # Get Packer's public key and inject it into cloud-init
  provisioner "shell-local" {
    inline = [
      "echo '[Security] Creating cloud-init configuration with Packer-generated SSH key'",
      "mkdir -p ./cloud-init",
      "cat > ./cloud-init/user-data << EOF",
      "#cloud-config",
      "users:",
      "  - name: rocky",
      "    sudo: ALL=(ALL) NOPASSWD:ALL",
      "    ssh_authorized_keys:",
      "      - \"{{ .SSHPublicKey }}\"",
      "    shell: /bin/bash",
      "lock_passwd: true",
      "chpasswd:",
      "  expire: false",
      "",
      "runcmd:",
      "  - systemctl enable --now sshd",
      "EOF",
      "echo '[Security] Cloud-init configuration created with Packer-generated public key'"
    ]
  }

  # Verification that SSH works
  provisioner "shell" {
    inline = [
      "echo 'SSH connection successful!'",
      "echo 'Connected as: '$(whoami)",
      "echo 'Hostname: '$(hostname)"
    ]
  }

  # Add log message about key deletion
  provisioner "shell-local" {
    inline = [
      "echo '[Security] Build completed at: $(date)'",
      "echo '[Security] Packer will automatically delete the ephemeral SSH key'"
    ]
  }
}