Packer QEMU build Windows Server

Hi all,

I am trying to build RAW type image of Windows server 2019, so I decided to try using QEMU builder.
The packer version is 1.8.3.
The problem I am facing is that WinRM fails to connect and I am thinking its related to qemu user network stack. The IP range is 10.0.2.0/24 and I have no way to connect to it from QEMU host. I have tried numerous options with qemuargs but ultimately failed to solve it.
Was also trying to find a way to utilize virbr0 or actual bridged interface but that does not seem to work with WIndows, Linux only.

Here is my main packer config file.

source "qemu" "testimage" {
  accelerator                      = "kvm"
  boot_wait                        = "10s"
  communicator                     = "winrm"
#  configuration_version            = "8.0"
  cpus                             = "${var.cpus}"
  memory                           = "${var.memory}"
  disk_size                        = "${var.disk_size}"
  disk_interface                   = "ide"
  net_device                       = "virtio-net"
#  net_bridge                       = "bridge0"
  qemu_binary                      = "/usr/bin/qemu-system-x86_64"
  qemuargs                         = [["-m", "${var.memory}"],
                                       ["-smp", "${var.cpus}"],
                             #         [ "-device", "virtio-net,netdev=user.0"],
                             #         [ "-netdev", "user,hostfwd=tcp::2222-:5985,id=forward"],
                             #         [ "-netdev", "user,id=user.0,", "hostfwd=tcp::2222-:5985", "net=192.168.122.1/24" ],
                             #         [ "-device", "virtio-net,netdev=user.0"],
                                        ["-vga", "qxl"]]
  floppy_files                     = ["${var.autounattend}",
                                        "./scripts/unattend.xml",
                                        "./scripts/domainjoin.ps1",
                  			"./scripts/sysprep.bat",
                                        "./scripts/enable-winrm.ps1",
                                        "./drivers/vioserial/2k19/amd64/*.cat",
                                        "./drivers/vioserial/2k19/amd64/*.inf",
                                        "./drivers/vioserial/2k19/amd64/*.sys",
                                        "./drivers/viostor/2k19/amd64/*.cat",
                                        "./drivers/viostor/2k19/amd64/*.inf",
                                        "./drivers/viostor/2k19/amd64/*.sys",
                                        "./drivers/NetKVM/2k19/amd64/*.cat",
                                        "./drivers/NetKVM/2k19/amd64/*.inf",
                                        "./drivers/NetKVM/2k19/amd64/*.sys",
                                        "./drivers/qxldod/2k19/amd64/*.cat",
                                        "./drivers/qxldod/2k19/amd64/*.inf",
                                        "./drivers/qxldod/2k19/amd64/*.sys",
                                        "./software/MetashapeFloating.lic",
                                        "./software/newrelic-infra.yml",
                                        "./software/flex-gpu.yml" ]

  cd_files                         = [ "${var.software_directory}" ]
  iso_checksum                     = "${var.iso_checksum}"
  iso_url                          = "${var.iso_url}"
  shutdown_command                 = "a:/sysprep.bat"
  shutdown_timeout                 = "10m"
  headless                         = var.headless
  format                           = "raw"
  vm_name                          = "testimage"
  winrm_password                   = "${var.password}"
  winrm_timeout                    = "${var.winrm_timeout}"
  winrm_username                   = "${var.username}"
  winrm_use_ssl                    = false
  winrm_insecure                   = true
  output_directory                 = "artifacts/qemu/${var.os_name}${var.os_version}"
}

build {
  sources = ["source.qemu.testimage"]

  provisioner "windows-shell" {
    execute_command = "{{ .Vars }} cmd /c \"{{ .Path }}\""
    scripts         = ["./scripts/dotnetfm-install.bat"]
    environment_vars = ["dotnetfm_package=${var.dotnetfm_package}"]
    valid_exit_codes = [ 0, 3010 ]
  }

  provisioner "windows-shell" {
    scripts = [ "./scripts/hosts-restore.bat", "./scripts/compact.bat"]
  }
}

I would really appreciate if someone was able to solve this as I am out of ideas.

The network on my Linux host, where I am running QEMU and trying to build images is:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp5s0f3u2u2c2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    link/ether 7a:30:6f:58:07:96 brd ff:ff:ff:ff:ff:ff
    inet 169.254.3.1/24 brd 169.254.3.255 scope link noprefixroute dynamic enp5s0f3u2u2c2
       valid_lft 772909sec preferred_lft 772909sec
    inet6 fe80::32c6:79c3:a940:8b16/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master bridge0 state UP group default qlen 1000
    link/ether 3c:ec:ef:71:87:fe brd ff:ff:ff:ff:ff:ff
9: bridge0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 3c:ec:ef:71:87:fe brd ff:ff:ff:ff:ff:ff
    inet 10.3.43.202/24 brd 10.3.43.255 scope global noprefixroute dynamic bridge0
       valid_lft 3316sec preferred_lft 3316sec
    inet6 fe80::3eec:efff:fe71:87fe/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
15: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:a4:fe:43 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
16: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:a4:fe:43 brd ff:ff:ff:ff:ff:ff

I was able to successfully build several different types of Linux distro but trying to build this windows server 2019 version with Winrm communicator is just not moving forward.

If you have any suggestions or need more information, please let me know.

I apologise upfront for a messy config, but as I mentioned, I tried several things.
I have strong feeling its network related, but I am out of ideas how to solve it.

Part of the log file:

    qemu.digrebar: 98.24% done, estimate finish Tue Oct 11 17:03:43 2022
    qemu.digrebar: 99.08% done, estimate finish Tue Oct 11 17:03:43 2022
    qemu.digrebar: 99.93% done, estimate finish Tue Oct 11 17:03:43 2022
    qemu.digrebar: Total translation table size: 0
    qemu.digrebar: Total rockridge attributes bytes: 0
    qemu.digrebar: Total directory bytes: 2048
    qemu.digrebar: Path table size(bytes): 26
    qemu.digrebar: Max brk space used 0
    qemu.digrebar: 590425 extents written (1153 MB)
    qemu.digrebar: Done copying paths from CD_dirs
2022/10/11 17:03:43 packer-builder-qemu plugin: [INFO] Creating disk with Path: artifacts/qemu/windows2019/digrebar and Size: 81440M
2022/10/11 17:03:43 packer-builder-qemu plugin: Executing qemu-img: []string{"create", "-f", "qcow2", "artifacts/qemu/windows2019/digrebar", "81440M"}
2022/10/11 17:03:43 packer-builder-qemu plugin: stdout: Formatting 'artifacts/qemu/windows2019/digrebar', fmt=qcow2 size=85396029440 encryption=off cluster_size=65536 lazy_refcounts=off
2022/10/11 17:03:43 packer-builder-qemu plugin: stderr:
2022/10/11 17:03:43 packer-builder-qemu plugin: Looking for available communicator (SSH, WinRM, etc) port between 2222 and 4444
2022/10/11 17:03:43 packer-builder-qemu plugin: Found available port: 2793 on IP: 127.0.0.1
==> qemu.digrebar: Found port for communicator (SSH, WinRM, etc): 2793.
2022/10/11 17:03:43 packer-builder-qemu plugin: Looking for available port between 5900 and 6000 on 127.0.0.1
==> qemu.digrebar: Looking for available port between 5900 and 6000 on 127.0.0.1
2022/10/11 17:03:43 packer-builder-qemu plugin: Found available port: 5947 on IP: 127.0.0.1
2022/10/11 17:03:43 packer-builder-qemu plugin: Found available VNC port: 5947 on IP: 127.0.0.1
2022/10/11 17:03:43 packer-builder-qemu plugin: Qemu --version output: QEMU emulator version 2.0.0, Copyright (c) 2003-2008 Fabrice Bellard
2022/10/11 17:03:43 packer-builder-qemu plugin: Qemu version: 2.0.0
==> qemu.digrebar: Starting VM, booting from CD-ROM
    qemu.digrebar: view the screen of the VM, connect via VNC without a password to
    qemu.digrebar: vnc://127.0.0.1:5947
    qemu.digrebar: The VM will be run headless, without a GUI. If you want to
    qemu.digrebar: view the screen of the VM, connect via VNC without a password to
    qemu.digrebar: vnc://127.0.0.1:5947
==> qemu.digrebar: Overriding default Qemu arguments with qemuargs template option...
2022/10/11 17:03:43 packer-builder-qemu plugin: Executing /usr/bin/qemu-system-x86_64: []string{"-boot", "once=d", "-fda", "/tmp/packer3370075535", "-machine", "type=pc,accel=kvm", "-smp", "8", "-vga", "qxl", "-name", "digrebar", "-netdev", "user,id=user.0,hostfwd=tcp::2793-:5985", "-vnc", "127.0.0.1:47", "-drive", "file=artifacts/qemu/windows2019/digrebar,if=ide,cache=writeback,discard=ignore,format=qcow2", "-drive", "file=/root/packer-marko/iso/win2019.iso,media=cdrom", "-drive", "file=/tmp/packer1988946110.iso,media=cdrom", "-m", "8192", "-device", "virtio-net,netdev=user.0"}
2022/10/11 17:03:43 packer-builder-qemu plugin: Started Qemu. Pid: 5312
2022/10/11 17:03:43 packer-builder-qemu plugin: Qemu stderr:
2022/10/11 17:03:43 packer-builder-qemu plugin: Qemu stderr: (process:5312): GLib-WARNING **: 17:03:43.827: gmem.c:489: custom memory allocation vtable not supported
==> qemu.digrebar: Waiting 10s for boot...
==> qemu.digrebar: Connecting to VM via VNC (127.0.0.1:5947)
2022/10/11 17:03:55 packer-builder-qemu plugin: Connected to VNC desktop: QEMU (digrebar)
==> qemu.digrebar: Typing the boot command over VNC...
    qemu.digrebar: Not using a NetBridge -- skipping StepWaitGuestAddress
==> qemu.digrebar: Using WinRM communicator to connect: 127.0.0.1
2022/10/11 17:03:55 packer-builder-qemu plugin: Waiting for WinRM, up to timeout: 2h0m0s
==> qemu.digrebar: Waiting for WinRM to become available...
2022/10/11 17:03:55 packer-builder-qemu plugin: [INFO] Attempting WinRM connection...
2022/10/11 17:03:55 packer-builder-qemu plugin: [DEBUG] connecting to remote shell using WinRM
==> qemu.digrebar: Waiting for WinRM to become available...
2022/10/11 17:03:55 packer-builder-qemu plugin: [INFO] Attempting WinRM connection...
2022/10/11 17:03:55 packer-builder-qemu plugin: [DEBUG] connecting to remote shell using WinRM
2022/10/11 17:05:10 packer-builder-qemu plugin: [ERROR] connection error: unknown error Post "http://127.0.0.1:2793/wsman": read tcp 127.0.0.1:59670->127.0.0.1:2793: read: connection reset by peer
2022/10/11 17:05:10 packer-builder-qemu plugin: [ERROR] WinRM connection err: unknown error Post "http://127.0.0.1:2793/wsman": read tcp 127.0.0.1:59670->127.0.0.1:2793: read: connection reset by peer
2022/10/11 17:05:15 packer-builder-qemu plugin: [INFO] Attempting WinRM connection...
2022/10/11 17:05:15 packer-builder-qemu plugin: [DEBUG] connecting to remote shell using WinRM

From VM perspective, the OS is installed and network configure, including WinRM service.
WinRM service script that runs right after installation:

Enable-PSRemoting -Force
winrm quickconfig -q
winrm quickconfig -transport:http
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="800"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
winrm set winrm/config/client/auth '@{Basic="true"}'
winrm set winrm/config/listener?Address=*+Transport=HTTP '@{Port="5895"}'
netsh advfirewall firewall set rule group="Windows Remote Administration" new enable=yes
netsh advfirewall firewall set rule name="Windows Remote Management (HTTP-In)" new enable=yes action=allow remoteip=any
Set-Service winrm -startuptype "auto"
Restart-Service winrm

The network on the test VM is configured with 10.0.2.15 (its the first address in user network stack dhcp range) and this is where the problem start.
If I do not set the winrm_host, I would expect that qemu is smart to forward or route, using some internal mechanism to reach the 10.0.2.0 network, but I was unable to find if and where it does it.
If I set the winrm_host parameter to 10.0.2.15, than message is no route to host, which does make sense, but how to solve it.

1 Like

Thanks for the technical explanation. Can you share your github code details of your Windows build?
I am also working on Windows Server Image Build from KVM. I am keep getting WinRM waiting to connect after timeout (2Hrs) it exited.
Thanks,
CHarly.R