Qemu/Ubuntu/Cloud-Init SSH Auth issues

Hello. I’ve been going around in circles trying to figure this one out and decided it’s time to just post even though I am sure it’s been brought up before.

I’m trying to use Packer to build Qemu images for Ubuntu 22.04.

My build machine is headless but I am able to watch the VM bootup via VNC.
So far as I can tell the autoinstall is running but the build inevitably fails when Packer attempts to connect with SSH.

I specifically get an auth failure, rather than timeout, etc so I’m fairly certain this is related to my cloud-init user data.

I’m going to paste the full hcl and user-data (with the password) because I don’t intend on actually using this password :smiley:

Any thoughts on how to solve this would be greatly appreciated.

kvm-ubuntu.pkr.hcl

packer {
    required_plugins {
        qemu = {
            version = ">= 1.0.3"
            source = "github.com/hashicorp/qemu"
        }
        ansible = {
            version = ">= 1.0.1"
            source = "github.com/hashicorp/ansible"
        }
    }
}

source "qemu" "test" {
    vm_name                 = "test"
    iso_url                 = "http://www.releases.ubuntu.com/22.04/ubuntu-22.04-live-server-amd64.iso"
    iso_checksum            = "sha256:84aeaf7823c8c61baa0ae862d0a06b03409394800000b3235854a6b38eb4856f"
    shutdown_command        = "sudo systemctl poweroff"
    memory                  = 2048
    disk_size               = "5000M"
    format                  = "qcow2"
    http_directory          = "http-server"
    net_device              = "virtio-net"
    disk_interface          = "virtio"
    boot_wait               = "3s"
    ssh_username            = "test"
    headless                = true
    vnc_bind_address        = "0.0.0.0"
    ssh_password            = "test"
    ssh_timeout             = "60m"
#    ssh_private_key_file    = "~/.ssh/id_ed25519"
    boot_command            = [
        "<esc><esc><esc><esc>e<wait>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del><del><del><del><del><del><del><del>",
        "<del>",
        "linux /casper/vmlinuz --- autoinstall ds=\"nocloud-net;seedfrom=http://{{.HTTPIP}}:{{.HTTPPort}}/\"<enter><wait>",
        "initrd /casper/initrd<enter><wait>",
        "boot<enter>",
        "<enter><f10><wait>",
    ]
}

build {
    sources = ["source.qemu.test"]

#    provisioner "ansible" {
#        playbook_file = "./playbook.yml"
#        roles_path    = "../roles"
#    }
}

http-server/user-data

#cloud-config
autoinstall:
  version: 1

  locale: en_US.UTF-8
  keyboard:
    layout: us

  identity:
    hostname: test
    username: test
    password: $6$rounds=4096$v7c.DGzyxCl7MvjD$bUZ1BehObKFyHhk1eiU28U8JJedhaMz9A.9hQldQ5py2wQPl/DMHuBiAliW2v6DoV4nTeBgNQ9xuXqQOQB1oR0

  ssh:
    allow-pw: true
    install-server: true

  packages:
    - qemu-guest-agent
    - cloud-initramfs-growroot
    - openssh-server

  user-data:
    timezone: UTC

  late-commands:
    - "echo 'test ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/test"
    - "chmod 440 /target/etc/sudoers.d/test"

terminal output

qemu.test: output will be in this color.

==> qemu.test: Retrieving ISO
==> qemu.test: Trying http://www.releases.ubuntu.com/22.04/ubuntu-22.04-live-server-amd64.iso
==> qemu.test: Trying http://www.releases.ubuntu.com/22.04/ubuntu-22.04-live-server-amd64.iso?checksum=sha256%3A84aeaf7823c8c61baa0ae862d0a06b03409394800000b3235854a6b38eb4856f
==> qemu.test: http://www.releases.ubuntu.com/22.04/ubuntu-22.04-live-server-amd64.iso?checksum=sha256%3A84aeaf7823c8c61baa0ae862d0a06b03409394800000b3235854a6b38eb4856f => /home/ing/.cache/packer/b9441068de828d36573e1274dfe77f69aebda15a.iso
==> qemu.test: Starting HTTP server on port 8110
==> qemu.test: Found port for communicator (SSH, WinRM, etc): 3827.
==> qemu.test: Looking for available port between 5900 and 6000 on 0.0.0.0
==> qemu.test: Starting VM, booting from CD-ROM
    qemu.test: The VM will be run headless, without a GUI. If you want to
    qemu.test: view the screen of the VM, connect via VNC without a password to
    qemu.test: vnc://0.0.0.0:5988
==> qemu.test: Waiting 3s for boot...
==> qemu.test: Connecting to VM via VNC (0.0.0.0:5988)
==> qemu.test: Typing the boot command over VNC...
    qemu.test: Not using a NetBridge -- skipping StepWaitGuestAddress
==> qemu.test: Using SSH communicator to connect: 127.0.0.1
==> qemu.test: Waiting for SSH to become available...
==> qemu.test: Error waiting for SSH: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain
==> qemu.test: Deleting output directory...
Build 'qemu.test' errored after 3 minutes 34 seconds: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

==> Wait completed after 3 minutes 34 seconds

==> Some builds didn't complete successfully and had errors:
--> qemu.test: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none password], no supported methods remain

==> Builds finished but no artifacts were created.

Supply an authorized-keys: block in your user-data.yaml under your ssh: block

ssh:
    install-server: true
    allow-pw: true
    authorized-keys:
      - ssh-rsa <encrypted_key> user@host

Fetch your key with:

cat ~/.ssh/id_rsa.pub

I fought this for about a month before giving up and moving to ssh.
Also, you shouldnt need anything in this block to get 22.04 up:

  packages:
    - qemu-guest-agent
    - cloud-initramfs-growroot
    - openssh-server

I found none of these packages to assist me.

I used the following for late commands:

    - "echo 'packer ALL=(ALL) NOPASSWD: ALL' > /target/etc/sudoers.d/test"
    - curtin in-target --target=/target -- chmod 440 /etc/sudoers.d/test

I mean… I can do this but I’m confused, this isn’t the same issue.

You’re providing an RSA key, I’m trying to get password auth working, these are entirely separate matters.

I would like to know why password auth is not working with Packer when obviously I know it works if I go and set it up manually.

openssh-server I probably do not need, it was added as a debugging step, the install server param should handle this but in fact the other 2 packages are needed if I want the root FS to match the full size of the image provided (there are other ways to do this), as for the guest-agent, it’s only needed if you are doing things that need it (this is particularly useful if you are using terraform to deploy your VMs as otherwise terraform is incapable of returning the VM’s IP address, unless something has changed there) :wink: