CentOS7 qemu VM Stuck After Configured

Hello,

I’ve been trying to create a basic CentOS7 qcow2 image. Packer spins up a VM from the ISO specified, and I can connect to the running instance thru VNC. When the VM gets to the point where it runs the post instructions, the VM just stops there, even with no post instructions in the kickstart file. Eventually Packer times out and I get:

==> kvm: Timeout waiting for SSH.

I assume the VM is supposed to reboot for Packer to ssh in. Any suggestions would be greatly appreciated.

Packer version: 1.5.1
OS: Ubuntu 18.04 LT

Packer template:

{
“variables”: {
“build_number”: “{{timestamp}}”,
“centos_arch”: “x86_64”,
“iso_url”: “http://mirror.cs.vt.edu/pub/CentOS/7.7.1908/isos/x86_64/CentOS-7-x86_64-Everything-1908.iso”,
“iso_checksum_url”: “http://mirror.cs.vt.edu/pub/CentOS/7.7.1908/isos/x86_64/sha256sum.txt”,
“iso_checksum_type”: “sha256”,
“username”: “root”,
“password”: “temp12345”,
“timeout”: “40m”,
“headless”: “true”,
“http”: “http”,
“ks_file”: “ks.cfg”,
“disk_size”: “22528”,
“memory”: “5120”,
“cpus”: “4”
},
“builders”: [
{
“name”: “kvm”,
“type”: “qemu”,
“iso_url”: “{{ user iso_url}}”,
“iso_checksum_url”: “{{user iso_checksum_url}}”,
“iso_checksum_type”: “{{user iso_checksum_type}}”,
“ssh_username”: “{{user username}}”,
“ssh_password”: “{{user password}}”,
“ssh_timeout”: “{{user timeout}}”,
“headless”: “{{user headless}}”,
“shutdown_command”: “shutdown -P now”,
“http_directory”: “{{ user http }}”,
“boot_command”: [
" text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/{{user ks_file}}"
],
“disk_size”: “{{ user disk_size }}”,
“memory”: “{{ user memory }}”,
“cpus”: “{{ user cpus }}”
}
]
}

Kickstart file:

# Install OS instead of upgrade
install
# Keyboard layouts
keyboard 'us'
# Root password
rootpw temp12345
# System timezone
timezone America/New_York
# Reboot after installation
reboot --eject
# poweroff
# System language
lang en_US.UTF-8
# Firewall configuration
firewall --enable --ssh
# System authorization information
auth  --useshadow  --passalgo=sha512
# Use graphical install
#graphical
text
firstboot --disabled
# SELinux configuration
selinux --disabled
# Networking
network  --bootproto=dhcp --onboot=on --device=eth0 --ipv6=auto
# Do not configure the X Window System
skipx
# Accept EULA without user interaction
eula --agreed
# System bootloader configuration
bootloader --location=mbr
# Partition clearing information
clearpart --all
zerombr
# Disk partitioning information
autopart
# User setup
user --name=centos --groups=centos --password=centos
# Services
services --enabled=NetworkManager,sshd
# Packages
%packages --nobase
@core
-*firmware
-b43-openfwwf
-efibootmgr
-audit*
-libX*
-fontconfig
-freetype
sudo
openssh-clients
openssh-server
make
perl
kernel-firmware
kernel-devel
%end

Can you please provide a logfile using
PACKER_LOG=1 PACKER_LOG_PATH=packer.log packer build your-template.json?

The log file is below. I’ve removed most of the attempts to connect thru ssh, but kept the top and the bottom portion of the log. I did cancel the build, so you’ll see those log entries reflect that towards the bottom. Thanks for the help!

packer.log.txt (16.3 KB)

Is it possible to root@127.0.0.1:2922 during build, and if, can you debug the network configuration in this session?

This hint is interesting, too:

%post
sed -i "s/#PermitRootLogin yes/PermitRootLogin yes/g" /etc/ssh/sshd_config
%end

Seems like a connection is established, but the authentication part is failing.

jorge@DESKTOP:~/Image-Build$ ssh -vvv -p 2922 -o PreferredAuthentications=password root@127.0.0.1
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
debug1: Reading configuration data /home/jorge/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving “127.0.0.1” port 2922
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 2922.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/jorge/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
ssh_exchange_identification: Connection closed by remote host

I looked for the /etc/sshd_config, but there seems to be only an anaconda file there (/etc/sshd_config.anaconda). I’m starting to think maybe my kickstart file is not complete…

Is it asking for a password? It seems the configuration is expecting a ssh-key authentication only.

Its not asking for a password when attempting to ssh in, it just fails. Even though I added the option for password authentication, and set the root password in the kickstart file. Yeah, seems like I may have to add steps to add a public key to /root/.ssh/authorized_keys, maybe as a %post directive?

Or try to figure out why it’s not pretending to expect a password. :wink: I’ve no experiences with Kickstart, because I am using presseding. But the %post-step sounds good.

Yea, i need to get better kickstart :slight_smile: I’ll see how the %post option goes. Thanks for the feedback!

You are welcome :+1: Hope we can fix that.

Ok, so after a couple of hours I found the issue. By default RedHat based systems do not setup ssh during installation by default. One must add the inst.sshd boot option which:

inst.sshd
Starts the sshd service during the installation, which allows you to connect to the system during the installation using SSH and monitor its progress. For more information on SSH, see the ssh(1) man page and the corresponding chapter in the Red Hat Enterprise Linux 7 System Administrator’s Guide. By default, sshd is only automatically started on IBM Z, and on other architectures, sshd is not started unless the inst.sshd option is used.

After updating my boot_command like this:

"boot_command": [
          "<tab> inst.sshd inst.text inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/{{user `ks_file`}}<enter><wait>"
        ],

Packer was able to ssh in. But, it connects prematurely and doesn’t give the installation a chance to complete…

So, I’m testing with enabling ssh during the %post phase in the kickstart file, acting as a sort of latch for Packer to ssh in.

Post installation tasks

%post
mv /etc/ssh/sshd_config.anaconda /etc/ssh/sshd_config
systemctl start sshd
%end

This was a fun one :stuck_out_tongue_winking_eye:

1 Like

Congrats!

Concerning the premature connect: https://packer.io/docs/templates/communicator.html#pause_before_connecting
Maybe you could give it an additional try. :blush:

1 Like