Elastic Beanstalk Environment Fails to Launch with Packer-Built AMI

Packer
1

Greetings Packer folks!

I’m facing an issue when creating a new Elastic Beanstalk (EBS) environment using a custom AMI built with Packer. Here’s the situation:

I started with an AWS-selected, publicly available AMI that Elastic Beanstalk suggests when creating a new environment via the wizard:

AMI: aws-elasticbeanstalk-amzn-2023.5.20240805.64bit-eb_php82_amazon_linux_2023-hvm-2024-08-08T01-16-12.699Z

I used this AMI with Packer to install our custom scripts, apps, and extensions, and then built a new AMI from it. However, when I try to launch an Elastic Beanstalk environment using this Packer-built AMI, the EC2 instances terminate continuously in a loop and the environment never stabilizes.

To troubleshoot, I manually launched an EC2 instance using the original AWS AMI, ran all our custom scripts manually, and created a new AMI from this instance. When I used this new manually-created AMI with Elastic Beanstalk, it worked perfectly, confirming that the problem lies with the AMI created via Packer.

Here’s what I’ve tried so far:

  • I built the AMI using both HCL2 and JSON configurations in Packer, but both resulted in the same looping termination issue.
  • I tested building from my local machine and also from Jenkins running on an AWS node, with identical results.
  • I even created a Packer-built AMI by simply adding a single text file without running any scripts, and this also failed with EBS (though it works fine when launching an EC2 instance directly via the EC2 dashboard).
  • I’ve checked CloudInit logs, and they seem to indicate that CloudInit is running normally.

Key Points:

  1. AMI works perfectly when an EC2 instance is launched directly.
  2. The issue only occurs when using the AMI with Elastic Beanstalk.
  3. The problem persists regardless of Packer configuration (HCL2/JSON).
  4. Manually configured AMIs work fine with EBS.
  5. I compared the Packer built AMI with Origin AMI in two different EC2s and noticed the cloud init logs are drastically different (1800 lines vs 900 lines).

I suspect Packer might be altering something in the AMI during the build process that causes the EBS launch to fail, but I’m unable to pinpoint the exact issue. Has anyone faced similar problems, or can provide insights on what might be going wrong?

Thanks in advance for any help or guidance!