Best Practise builds in Packer


In scope of our current set up, we are looking at the setup of Packer and Terraform to full provision our servers in a declarative and immutable way. Our understanding of Packer is to create new versions for every new flavor, so the provisioning step only needs to destroy and recreate the server. For this I had the following questions:

Balance between provisioning vs build declaration
In its extreme, you would build a new image for every little change, and I have the feeling you want to declare as much as possible as makes sense in the build step of Packer. However, as we are in the early stages and currently still use Ansible to provision everything, I was wondering how much of the declaration make sense to migrate to a Build step vs a Provisioning Step.

Chaining Packer Builds
As with the previous assumption of creating images based on any infrastructure change, we would like to chain certain Packer build in-order to realize DRY declarations. This would, for example, be used in the following Nomad declarations:

Base Ubuntu → Nomad Client → Nomad Client Ingress

Using the Build Source block, a ‘run-time’ source definition, we could use a previous created build as packer build runs the build from top to bottom (see Allow HCL source ... inheritence? merging? templating? · Issue #9154 · hashicorp/packer · GitHub).

I was wondering if this is a common practice within the philosophy of Packer, or if you’d want one base image to be provisioned for all. I also noticed, with the above strategy, validations might not fully work. For example, with an empty source block that is filled in the build-source block. I was hoping these two blocks were squashed in the config parsing.

Image management
Now that we would manage a couple of images, I also notice a coupled set-up between the Packer image names and the Terraform image names you’d use. This also makes me think that I might need to move more to the Terraform provisioning step, so every new flavor is provisioned only in Terraform instead of also build within Packer.

Hopefully our current approach to Terraform and Packer is clear, and I hope anyone can share its ideas about these assumptions.