Packer v1.7.10
This just started recently where sysprep isn’t completing correctly. It appears that the SysprepInstance.ps1 -NoShutdown is not completing. This is causing the new AMI to break. The AMi is in a unrecoverable state.
This is the last line of the packer script.
{
“type”: “powershell”,
“inline”: [
“C:/ProgramData/Amazon/EC2-Windows/Launch/Scripts/InitializeInstance.ps1 -Schedule”,
“C:/ProgramData/Amazon/EC2-Windows/Launch/Scripts/SysprepInstance.ps1 -NoShutdown”
]
Strangely enough, I’m seeing the same behavior in my environment, in AWS, when doing the sysprep step through Ansible over Packer. I’ve been thinking of switching to using the powershell provisionier, via Packer directly, just like you mention in your example. For me, things worked just fine until maybe mid-January.
Yes, I run builds monthly and it decided to stop working in Feb. The very strange thing is that when I use the 2019 Windows image it works fine. Some reason 2016 just does not work. I’ve been trying older versions of Packer and those don’t seem to work.
For me, this happens on Windows Server 2019. I’m no longer building 2016, so I can’t say how it goes there. The other strange thing is that after a few failed attempts, it suddenly works, then it starts failing again, then it works for once, and so on.
Did anyone ever find a resolution to this? Can confirm we’re seeing it on the most recent AWS AMIs for Windows Server 2016 and 2019
I found if the Windows instance needs a reboot due to the configuration of the AMI, Sysprep fails. So I added a “reboot if required” at the end of my Ansible playbook, with a 2 min delay to allow the instance to be completely restarted before running Sysprep, then it worked again. I also found some articles about updates causing issues but did not have to dig into that.
Changing the instance type did the trick for me. All builds completed successfully when I went with a different instance type.
Here’s what I had to do to get it to work. Sometimes just calling sysprep with the generalize/oobe would work, other times it wouldn’t. I think it was a timing thing on when it closed sysprep vs. when it was actually completed.
{
"type": "powershell",
"elevated_user": "SYSTEM",
"elevated_password": "",
"inline": [
"echo '>>> Sysprepping VM ...'",
"If ((Get-Item \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\WindowsStore\").Property -contains \"DisableWindowsConsumerFeatures\"){Remove-ItemProperty -Path \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\WindowsStore\" -Name \"DisableWindowsConsumerFeatures\"}",
"If ((Get-Item \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\WindowsStore\").Property -contains \"AutoDownload\"){Remove-ItemProperty -Path \"HKLM:\\SOFTWARE\\Policies\\Microsoft\\WindowsStore\" -Name \"AutoDownload\"}",
"Start-Process -FilePath \"$Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe\" -ArgumentList \"/oobe /generalize /quiet /quit\" -Wait -NoNewWindow",
"while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Verbose $imageState.ImageState -Verbose; Start-Sleep -s 15 } else { break } }"
]
}],
Has anyone noticed this happening again? I’m trying to run the provided AWS EC2 Launch script called SysprepInstance.ps1 with the -NoShutdown flag and it’s apparently returning 1 instead of 0, though it’s not printing any useful output.
This is happening on our 2016, 2019, and 2022 builds.
EC2Launch has new version 1.3.2003961 released on 6 December 2022 and SysprepInstance.ps1 is failing in it without any output. We raised a ticket with AWS. Until it’s fixed you can update your scripts to use older version - 1.3.2003923.
Thanks Ivan, I followed your suggestion and downgrading to version “1.3.2003923” indeed fixed it.
For anyone looking on how to install, you can follow this aws doc: Install the latest version of EC2Launch - Amazon Elastic Compute Cloud
and substitute the “latest” with the version you want to install.
For example:
https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/1.3.2003923/EC2-Windows-Launch.zip
Amazon’s documentation on this isn’t great, but I believe invoking the PS1 scripts is EC2Launchv1. Here’s how you sysprep in EC2Launchv2:
provisioner "powershell" {
inline = [
#Sysprep the instance with ECLaunch v2. Reset enables runonce scripts again.
"Set-Location $env:programfiles/amazon/ec2launch",
"./ec2launch.exe reset -c -b",
"./ec2launch.exe sysprep -c -b"
]
}
I got same issue, i tried downgrading the version but still my packer build is failing
"type": "powershell",
"pause_before": "10s",
"inline": [
"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\InitializeInstance.ps1 -Schedule",
"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SysprepInstance.ps1 -NoShutdown"
]
},
Here is a powershell provisioner I added right before the provisioner that calls the SysPrep powershell scripts. This is working for me.
{
"type": "powershell",
"elevated_user": "<Your_Admin_Username>",
"elevated_password": "<Your_Admin_Password>",
"max_retries": 5,
"inline": [
"$ErrorActionPreference='Stop'",
"Write-Host 'Rollback to version 1.3.2003923 of EC2Launch v1'",
"Copy-Item -Path \"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Config\" -Destination \"$env:USERPROFILE\\Desktop\\EC2Launch_Config_Backup\" -Recurse",
"Remove-Item -Recurse -Force \"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\"",
"$Ec2LaunchVersion = \"1.3.2003923\"",
"mkdir $env:USERPROFILE\\Desktop\\EC2Launch",
"$Url = \"https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/$Ec2LaunchVersion/EC2-Windows-Launch.zip\"",
"$DownloadZipFile = \"$env:USERPROFILE\\Desktop\\EC2Launch\\\" + $(Split-Path -Path $Url -Leaf)",
"Invoke-WebRequest -Uri $Url -OutFile $DownloadZipFile",
"$Url = \"https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/$Ec2LaunchVersion/install.ps1\"",
"$DownloadZipFile = \"$env:USERPROFILE\\Desktop\\EC2Launch\\\" + $(Split-Path -Path $Url -Leaf)",
"Invoke-WebRequest -Uri $Url -OutFile $DownloadZipFile",
"& $env:USERPROFILE\\Desktop\\EC2Launch\\install.ps1",
"Copy-Item -Path \"$env:USERPROFILE\\Desktop\\EC2Launch_Config_Backup\\*\" -Destination \"C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Config\" -Recurse -Force"
]
},
Thanks Michael, your provisioner script fixed me as well! I opened a case with AWS support on this issue and was told an EC2Launch fix is in the works.
This should be fixed now. AWS just reached back out to the support ticket I had opened and the new version is 1.3.2003975:
https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/1.3.2003975/EC2-Windows-Launch.zip
They did mention the documentation has not been updated yet (EC2Launch version history)
Any AWS AMIs that include the package by default will likely not be updated until January.
Confirmed fix is included in Windows_Server-2019-English-Full-Base-2022.12.28 AMI
I had to use the following script
mkdir $env:USERPROFILE\Desktop\EC2Launch
$Url = “https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/1.3.2003923/EC2-Windows-Launch.zip”
$DownloadZipFile = "env:USERPROFILE\\Desktop\\EC2Launch\\" + (Split-Path -Path $Url -Leaf)
Invoke-WebRequest -Uri $Url -OutFile $DownloadZipFile
$Url = “https://s3.amazonaws.com/ec2-downloads-windows/EC2Launch/1.3.2003923/install.ps1”
$DownloadZipFile = "env:USERPROFILE\\Desktop\\EC2Launch\\" + (Split-Path -Path $Url -Leaf)
Invoke-WebRequest -Uri $Url -OutFile $DownloadZipFile
& $env:USERPROFILE\Desktop\EC2Launch\install.ps1