I have created a packer template to build macOS 11 Big Sur VMs for testing. It’s also the first complex template I have created using hcl syntax. As enterprises are ramping up their testing macOS Big Sur several questions around packer usage and related techniques have come up. Let’s discuss those questions here so others can benefit as well.
The git repo for this template is here.
https://github.com/trodemaster/packer-macOS-11
Been getting an odd error about my shasum… but then ran a packer validate macOS_11.pkr.hcl and basically it can’t find any of the linked files within the template.
Error: 1 error(s) occurred:
* invalid checksum: encoding/hex: invalid byte: U+0069 'i' in "install_bits/macOS_1101_installer.shasum"
on macOS_11.pkr.hcl line 85:
(source code not available)
Error: 1 error(s) occurred:
* invalid checksum: encoding/hex: invalid byte: U+0069 'i' in "install_bits/macOS_1101_installer.shasum"
on macOS_11.pkr.hcl line 167:
(source code not available)
Error: 1 error(s) occurred:
* source_path is invalid: stat output/macOS_11_base/macOS_11_base.vmx: no such file or directory
on macOS_11.pkr.hcl line 226:
(source code not available)
Sounds like you have not built/downloaded the prerequisites yet. Have a look at https://github.com/trodemaster/packer-macOS-11#prerequisite-installer-bits
Apple being Apple does not simply post nice .iso files of their OS and compiler publicly. The prerequisites script wrapps a couple other mac comunity tools which generate the needed installer .iso and create a checksum. If you want Xcode then you will need to download them from developer.apple.com and put the files into the install_bits dir firs.
This template has three builders inside it. The main one full does the full build of the OS including installing Xcode. With the base build it’s just the OS and VMware tools that get installed. When you run the customize build it starts with the output of the base build. If you have never built the base then the errors about missing .vmx files with show on validate.
If you get success with the scripts/buildprerequs.sh then run
packer build -only=base.vmware-iso.macOS_11_base macOS_11.pkr.hcl
you should be off to a good start.
I have no error at all on building my iso file and all ready, but when the VM start it’s automatically going to the the boot manager.
What is the best way of troubleshoot.
I tried to do all the step of the ISO creation one by one but it’s still the same result.
The only thing who could cause it, is I’m building an iso file of 11.0.1 but I’m running all on a host who is 11.1 beta (last)… could it block something?
mcnahum make sure your on Fusion 12.1. Versions of the host should not matter. I’m unclear on what you mean by boot manager? Try these suggestions and post a screen shot if they don’t resolve the issue.
Boot timing can be affected by the host system and disk speed. If packer starts to send boot commands before the OS is fully running nothing useful will happen after it boots. Try doubling the value for boot wait. You can watch the VM boot and note the timing of packer logging " Typing the boot command over VNC" and if macOS is at the booted up and Recovery app is up.
CPU and RAM values can be tuned by adjusting a couple of variables. The simplest way is to pass them on the packer commandline. If the host system is starved for resources it will mess with the boot timing.
packer build -force -only=base.vmware-iso.macOS_11_base -var cpu_count=2 -var ram_gb=6 macOS_11.pkr.hcl
Thanks, everything in the directory…
packer just doesn’t see
Error: 1 error(s) occurred:
* invalid checksum: encoding/hex: invalid byte: U+0069 'i' in "install_bits/macOS_1101_installer.shasum"
on macOS_11.pkr.hcl line 85:
(source code not available)
Error: 1 error(s) occurred:
* invalid checksum: encoding/hex: invalid byte: U+0069 'i' in "install_bits/macOS_1101_installer.shasum"
on macOS_11.pkr.hcl line 167:
(source code not available)
Error: 1 error(s) occurred:
* source_path is invalid: stat output/macOS_11_base/macOS_11_base.vmx: no such file or directory
on macOS_11.pkr.hcl line 226:
(source code not available)
Not sure why it doesn’t see the files
I have used packer before and have been converting my older templates to the new hcl version.
I know its not finding the paths… if I put none in the shasum within template so it skips the checksum… it goes on to the ISO file and dies not finding file
One of the errors is implying that packer is trying to use the path string as the checksum itself. This is why the default vault starts with file: to specify that the checksum is stored in a file. The iso_filename variable value is only a path. Have you modified the variable default values at all? Are you 100% sure that you are running the packer command from the root of the git repo? EG the following succeeds from the same path that your running packer from?
ls install_bits/macOS_1110_installer.shasum
Alternatively, you can specify the shasum and the absolute path to the iso on the command line.
packer build -force -only=base.vmware-iso.macOS_11_base -var iso_filename=/Users/blake/packer-macos-11/install_bits/macOS_1101_installer.iso -var iso_file_checksum=sha256:7319b89db22a6e0dabbbb40b0a9899d93cb571645152985d3204c4f52b399438 macOS_11.pkr.hcl
Seems like something external to packer and this template is causing the issue you are seeing. You should investigate things like macOS access controls and maybe test giving the packer binary full disk access? Big Sur has lots of new restrictions and you could be hitting those as well.
Thanks! Using absolute paths work… something with PPPC I assume
Thanks
I’m using Vagrant + VirtualBox, while this repo uses VMware fusion. Is it easy to switch from fusion to virtualbox with most the code in this repo? Thanks a lot.
From Support virtualbox · Issue #7 · trodemaster/packer-macOS-11 · GitHub
Changing or adding a builder for packer is not difficult and you should be able to fork the repo and make those changes as needed. I would however test that the hypervisor your looking at actually supports the target guest OS first. With virtualbox you can try to manually build a vm and confirm it installs and networking functions before customizing the packer template.
Depending on your usage VMware Fusion is offered for free now. VMware Fusion Player – Personal Use License
I have the same problem, it enters recovery mode (in Turkish, no idea why) and stays there forever.
The logs from the command are these:
packer-macOS-11 git:(main) ✗ packer build -force -only=base.vmware-iso.macOS_11_base macOS_11.pkr.hcl
base.vmware-iso.macOS_11_base: output will be in this color.
==> base.vmware-iso.macOS_11_base: Retrieving ISO
==> base.vmware-iso.macOS_11_base: Trying install_bits/macOS_1110_installer.iso
==> base.vmware-iso.macOS_11_base: Trying install_bits/macOS_1110_installer.iso?checksum=sha256%3A9f6e5e3fb30d11c4d5f4f064ac881e495d16a6308f1e9c1ff3abe9ae8ea410fe
==> base.vmware-iso.macOS_11_base: install_bits/macOS_1110_installer.iso?checksum=sha256%3A9f6e5e3fb30d11c4d5f4f064ac881e495d16a6308f1e9c1ff3abe9ae8ea410fe => /Users/beat/code/packer-macOS-11/install_bits/macOS_1110_installer.iso
==> base.vmware-iso.macOS_11_base: Configuring output and export directories…
==> base.vmware-iso.macOS_11_base: Creating required virtual machine disks
==> base.vmware-iso.macOS_11_base: Building and writing VMX file
==> base.vmware-iso.macOS_11_base: Starting HTTP server on port 8969
==> base.vmware-iso.macOS_11_base: Starting virtual machine…
==> base.vmware-iso.macOS_11_base: Connecting to VNC…
==> base.vmware-iso.macOS_11_base: Waiting 3m0s for boot…
==> base.vmware-iso.macOS_11_base: Typing the boot command over VNC…
==> base.vmware-iso.macOS_11_base: Waiting for SSH to become available…
Versions used:
Host os: Catalina 10.15.7 (19H114)
VMWare Fusion Personal 12.1
Packer 1.6.6 (homebrew)
Trying again the same command it is stuck at the same screen, but this time in Portugese (Brazil). Command outputs are the same as before…
How can I debug? Happy to help to get this sorted out…
Have you tried increasing the boot wait in the builder you are using? The timing based automation often needs adjusting to match the build machine your using. Maybe add 30 seconds and retry.
Thanks, although increasing the boot wait time specifically didn’t solve my problem, it pointed me to dig deeper though. I increased the boot_key_interval to 200 ms, apparently my system was not fast enough to handle that (and this was most likely the reason the recovery mode was started in a “random” langauge). Now I get much further, but the installation of the VMWare Tools now fails, and thus failing the creation of the image.
My HD free space is around 220 GBs
Following the log, any hints to this? Again, happy to dig deeper if I know where to start…
➜ packer-macOS-11 git:(main) ✗ packer build -force -only=base.vmware-iso.macOS_11_base macOS_11.pkr.hcl
base.vmware-iso.macOS_11_base: output will be in this color.==> base.vmware-iso.macOS_11_base: Retrieving ISO
==> base.vmware-iso.macOS_11_base: Trying install_bits/macOS_1110_installer.iso
==> base.vmware-iso.macOS_11_base: Trying install_bits/macOS_1110_installer.iso?checksum=sha256%3A9f6e5e3fb30d11c4d5f4f064ac881e495d16a6308f1e9c1ff3abe9ae8ea410fe
==> base.vmware-iso.macOS_11_base: install_bits/macOS_1110_installer.iso?checksum=sha256%3A9f6e5e3fb30d11c4d5f4f064ac881e495d16a6308f1e9c1ff3abe9ae8ea410fe => /Volumes/SSD/packer-macOS-11/install_bits/macOS_1110_installer.iso
==> base.vmware-iso.macOS_11_base: Configuring output and export directories…
==> base.vmware-iso.macOS_11_base: Creating required virtual machine disks
==> base.vmware-iso.macOS_11_base: Building and writing VMX file
==> base.vmware-iso.macOS_11_base: Starting HTTP server on port 8943
==> base.vmware-iso.macOS_11_base: Starting virtual machine…
==> base.vmware-iso.macOS_11_base: Connecting to VNC…
==> base.vmware-iso.macOS_11_base: Waiting 10m0s for boot…
==> base.vmware-iso.macOS_11_base: Typing the boot command over VNC…
==> base.vmware-iso.macOS_11_base: Waiting for SSH to become available…
==> base.vmware-iso.macOS_11_base: Connected to SSH!
==> base.vmware-iso.macOS_11_base: Pausing 2m0s before the next provisioner…
==> base.vmware-iso.macOS_11_base: Provisioning with shell script: scripts/os_settings.sh
==> base.vmware-iso.macOS_11_base: Uploading /Applications/VMware Fusion.app/Contents/Library/isoimages/darwin.iso => ~/darwin.iso
base.vmware-iso.macOS_11_base: darwin.iso 3.24 MiB / 3.24 MiB [====] 100.00% 0s
==> base.vmware-iso.macOS_11_base: Provisioning with shell script: scripts/vmw_tools.sh
base.vmware-iso.macOS_11_base: /dev/disk3 Apple_partition_scheme
base.vmware-iso.macOS_11_base: /dev/disk3s1 Apple_partition_map
base.vmware-iso.macOS_11_base: /dev/disk3s2 Apple_HFS /Volumes/VMware Tools
base.vmware-iso.macOS_11_base: installer: Package name is VMware Tools
base.vmware-iso.macOS_11_base: installer: Installing at base path /
base.vmware-iso.macOS_11_base: installer: The install failed. (The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance. An error occurred while running scripts from the package “VMware Tools.pkg”.)
base.vmware-iso.macOS_11_base: “/Volumes/VMware Tools” unmounted successfully.
==> base.vmware-iso.macOS_11_base: Gracefully halting virtual machine…
base.vmware-iso.macOS_11_base: Waiting for VMware to clean up after itself…
==> base.vmware-iso.macOS_11_base: Deleting unnecessary VMware files…
base.vmware-iso.macOS_11_base: Deleting: output/macOS_11_base/macOS_11_base.plist
base.vmware-iso.macOS_11_base: Deleting: output/macOS_11_base/startMenu.plist
base.vmware-iso.macOS_11_base: Deleting: output/macOS_11_base/vmware.log
==> base.vmware-iso.macOS_11_base: Compacting all attached virtual disks…
base.vmware-iso.macOS_11_base: Compacting virtual disk 1
==> base.vmware-iso.macOS_11_base: Cleaning VMX prior to finishing up…
base.vmware-iso.macOS_11_base: Detaching ISO from CD-ROM device sata0:0…
base.vmware-iso.macOS_11_base: Disabling VNC server…
==> base.vmware-iso.macOS_11_base: Skipping export of virtual machine…
Build ‘base.vmware-iso.macOS_11_base’ finished after 3 hours 40 minutes.==> Wait completed after 3 hours 40 minutes
==> Builds finished. The artifacts of successful builds are:
→ base.vmware-iso.macOS_11_base: VM files in directory: output/macOS_11_base
➜ packer-macOS-11 git:(main) ✗
That is an installer error and is expected. VMware tools itself is installed but you will need to manually approve some security settings and reboot. Hopefully VMware will update tools to address this error. As for approving the needed security settings they can be set by installing a profile. Unfortunately apple disabled the ability to install profiles via cli in macOS 11.x so incorporating that into the packer build isn’t going to happen easily. If the community can solve that problem I’ll gladly add the changes to this template.
Ah, I see, thanks a lot for pointing that out. I somehow interpreted that “failed” line as a failure for creating the image 
But it’s all there. Thanks a lot for your work on that, it will be a life saver!
I have pushed some updated to the 11.2 beta branch that should address a lot of the timing issues people are seeing. Once Apple releases the final bits for macOS 11.2 I’ll validate and bump the main branch shortly after.
Check out Debugging - Other | Packer by HashiCorp to enable verbose logging with the packer build. VMware may output more details when run that way. I wouldn’t normally use sudo with packer either. Seems likely your running into some macOS system restrictions.
