Parallel Build Question

Packer
1

I am using Packer to build different OS images, targeting different platforms and different versions of each OS.

I have automation in place to allow me to build all of these images at the same time on the same physical machine.

When I kick off my builds (building 9 different images), I noticed that Packer is only building 1 image at a time and queues the others. When one of the builds completes, Packer starts building the next image. This behavior continues until all images have been created.

Here is the build command I am using:

  • packer build -on-error=abort config.json

Each build happens in a separate space on the OS, with it’s own files/configs.

Documentation for the build command suggests that Packer will build in parallel by default (-parallel-builds=0).

My assumption is that running in parallel also means concurrently. Is my assumption wrong? Is there a way to force Packer to build in parallel and concurrently?

Build Environment:

  • Packer Version:1.5.1
  • RHEL 7.7
  • 96 cores
  • 384GB Ram
2

After further investigation, I figured out why I couldn’t build in parallel.

My configuration files are written in JSON.

If I convert my configuration files to HCL, then I can concurrently build all of my images in parallel.

Hope this helps someone else.

1 Like
3

After further, further review, my previous workaround of using hcl was misguided and lucky.

The real reason I could not run packer in parallel was because my JSON configuration file was setting the http port to specifically be 8080. Since 2 applications can’t listen on the same port, my packer would run in series.
Here is what I did wrong:
"http_port_min": 8080,
"http_port_max": 8080,

The fix is trivial; either get rid of the min/max http port settings or set a range like this:
"http_port_min": 8000,
"http_port_max": 8080,

The only minor issue remaining is to open up ports on your host machines firewall to allow packer traffic.

Thanks to my collegue Jeremy → He deserves a raise! :wink:

1 Like
4

Good point! I typically use the defaults for http ports, so haven’t run into this exactly. But i have recently asked the “why aren’t my sources being built in parallel?” question, which for me was caused by using the -debug arg, which forces serial builds.