Building UEFI images with QEMU/KVM

Hi!

I’m trying to build qcow2 images for bare metal deployments (afterwards converted to raw) with using Packer and QEMU/KVM.

For BIOS supported hardware it works already fine and I’m building images for several Linux and Windows distributions. But since our newly DELL servers only supporting UEFI when using our NVME disks I need also UEFI images.

After hours of debugging I finally got KVM running with UEFI by using these qemuargs in Packer.

"qemuargs": [
        ["-m", "2048M"],
        ["-machine", "q35,accel=kvm"],
        ["-smp", "2"],
        ["-global", "driver=cfi.pflash01,property=secure,value=on"],
        ["-drive", "file=artifacts/qemu/{{user `name`}}/packer-{{user `name`}},if=virtio,cache=none,discard=unmap,format=qcow2"],
        ["-drive", "file=/usr/share/OVMF/OVMF_CODE.secboot.fd,if=pflash,format=raw,unit=0,readonly=on"],
        ["-drive", "file=/var/lib/libvirt/qemu/nvram/f20-uefi_VARS.fd,if=pflash,format=raw,unit=1"],
        ["-boot", "order=c,once=d,menu=on,strict=on"]
      ],

So far, so good. In my example I’m building a Ubuntu 20.04 image. The unattended installer is running fine and then Ubuntu installation reboots for the final provisioning steps.

And here comes the issue, the UEFI driver ignores the KVM -boot order and always using CD-ROM as first boot device. So it never boots from the installed qcow2 disk - so the machine hangs in a CD-ROM boot-loop forever.

Has someone already found a solution for this? I’ve been trying around with using bootorder on the -drive parameters but this seems also not being supported:

2020/11/25 05:22:52 packer-builder-qemu plugin: Qemu stderr: qemu-kvm: -drive file=artifacts/qemu/ubuntu-20.04.1/packer-ubuntu-20.04.1,if=virtio,cache=none,discard=unmap,format=qcow2,bootindex=0: Block format ‘qcow2’ does not support the option ‘bootindex’

Thank you in advance!

Alex

Oh man have I been down a rabbit hole this morning trying to solve this exact same problem!

Hopefully here’s a teach a man to fish moment…

The bootindex property only applies to -device definitions.

According to this https://bugzilla.redhat.com/show_bug.cgi?id=1752838#c2, OVMF will respect bootindex

Therefore we have to “override” using the qemuargs and define the proper -device arguments:

    "builders": [
      {
        "type": "qemu",
        "qemuargs": [
          [ "-pflash", "OVMF-pure-efi.fd" ],
          [ "-device", "virtio-scsi-pci" ],
          [ "-device", "scsi-hd,drive=drive0,bootindex=0" ],
          [ "-device", "scsi-cd,drive=cdrom0,bootindex=1" ]
        ],
        "disk_interface": "virtio-scsi",
        "cdrom_interface": "virtio-scsi", 
        // snip
      }
    ],

Theoretically the above SHOULD work, and allow you to make use of Packer’s magic for the actual -drive stanzas, however there’s an issue with the way Packer generates the cdrom drive. It contains an index=0 property and QEMU aborts due to the device already existing.

qemu-system-x86_64: -drive file=packer_cache/<HASH>.iso,if=none,index=0,id=cdrom0,media=cdrom: drive with bus=0, unit=0 (index=0) exists

I tried what I could think of to get it to work without having to redefine the -drives, but have so far been unsuccessful.


Hack-ish Workaround

If we redefine the -drive arguments, just as Packer generates them, and simply remove the index=0 from the cdrom drive, everything works as expected! … at least on my machine :slight_smile:

    "builders": [
      {
        "type": "qemu",
        "qemuargs": [
          [ "-pflash", "OVMF-pure-efi.fd" ],
          [ "-device", "virtio-scsi-pci" ],
          [ "-device", "scsi-hd,drive=drive0,bootindex=0" ],
          [ "-device", "scsi-cd,drive=cdrom0,bootindex=1" ],
          [ "-drive",  "if=none,file=output-qemu/packer-alpine,id=drive0,cache=writeback,discard=ignore,format=raw" ],
          [ "-drive",  "if=none,file=packer_cache/<HASH>.iso,id=cdrom0,media=cdrom" ]
        ],      
        "headless": true,
        "iso_url": "{{ user `mirror` }}/v{{ user `version` }}/releases/{{ user `arch` }}/alpine-{{ user `release` }}-{{ user `version` }}.{{ user `patch` }}-{{ user `arch` }}.iso",
        "iso_checksum": "file:{{ user `mirror` }}/v{{ user `version` }}/releases/{{ user `arch` }}/alpine-{{ user `release` }}-{{ user `version` }}.{{ user `patch` }}-{{ user `arch` }}.iso.sha256",
        "disk_interface": "virtio-scsi",
        "cdrom_interface": "virtio-scsi",
        "http_directory": "http",
        "vm_name": "packer-alpine",
        "disk_size": "10G",
        "format": "raw",
        "ssh_username": "root",
        "ssh_private_key_file": "~/.ssh/id_packer",
        "boot_wait": "1m",
        "boot_command": [
            // snip
        ],
        "shutdown_command": "poweroff",
        "shutdown_timeout": "1m"
      }
    ],

If I do manage to figure out how to keep the auto-generated -device arguments from Packer, I will report back!

Matthew