This is tangential, but why would you want to save an artifact that fails the tests? I typically use the Inspec provisioner as part of the same template, after the “main” provisioning. If the Inspec tests fail I don’t want to keep the artifact, but need to go back and fix the provisioner responsible for making the test pass.
We do the same thing with Trivy – if there’s a vulnerability found in the image, I want to abort the build and address the vulnerability.
I’m aware that there are probably reasons for the way you’re doing things now, but in the interest of “shifting left”, perhaps it might make sense to include the Inspec provisioner in the same template that creates the artifact?
Yep, make sense. I don’t want to store the artifact. This build is part of a jenkins pipeline so if the inspec tests fail, the original provisioner/template manifest is not released.
You have given me food for thought, though. If inspec confirms the build is bad, I would have to remove those artifacts somehow anyways. Hmm… I may end up putting them in the same template like you suggest.
As for why I don’t have it in the same template as the original provisioner right now:
In the past, I’ve had problems on new instances/nodes with packages like google-compute-engine making changes on boot if they haven’t been removed. So inspec in the same template would pass, but fail in a standalone template. Since we use multiple providers, there’s no knowing what the provider may install in their (Debian) image. With the standalone inspec provisioner, these types of things are easier to catch.
So ideally, I would like to test the image on a new instance or at least - if on the provisioner instance - after a power cycle, if I make sense.
We have our Inspec as a provisioner after the main install, but we do reboot the instance between. You can just have a shell provisioner that reboots and the following step waiting for the SSH to reappear.