I am struggling to figure out how to make the Packer Ansible provisioner connect to the qemu instance that has been created by Packer. FWIW, I have been using Packer to create and provision instances using the Ansible plugin for instance in OCI. The primary difference being that the instances in OCI have a network interface to which the provisioning node can connect.
This is the build file…
build {
sources = [
"virtualbox-iso.x86-64",
"qemu.x86-64",
"qemu.aarch64",
]
provisioner "file" {
source = var.packer_files
destination = "/tmp"
}
provisioner "shell" {
inline = ["hostnamectl; cat /etc/os-release"]
}
provisioner "ansible" {
ssh_host_key_file = var.ssh_private_key_file
user = "root"
extra_arguments = ["-vvvv", "--scp-extra-args", "'-O'"]
galaxy_file = "/workspace/oracle-linux/oracle-linux-image-tools/ansible/requirements/galaxy.yml"
inventory_directory = "/workspace/oracle-linux/oracle-linux-image-tools/ansible/inventory/"
keep_inventory_file = "true"
playbook_file = "/workspace/oracle-linux/oracle-linux-image-tools/ansible/site.yml"
}
provisioner "shell" {
script = var.provision_script
environment_vars = [
"OLIT_ACTION=provision",
]
}
provisioner "file" {
only = local.get_build_info
direction = "download"
source = "${var.build_info}/*"
destination = "${local.output_directory}/"
}
provisioner "shell" {
script = var.provision_script
environment_vars = [
"OLIT_ACTION=seal",
]
}
}
When Packer populates the ansible inventory file, it uses a port that is not the ssh port allocated during the instantiation of the qemu instance.
default ansible_host=127.0.0.1 ansible_user=root ansible_port=34653
When packer starts the vm, it allocates an ssh port as can be seen below; it is possible to connect to the port and login to the instance.
i.e.
2023/09/04 08:12:28 ui: ==> qemu.x86-64: Starting HTTP server on port 8235
2023/09/04 08:12:28 ui: ==> qemu.x86-64: Found port for communicator (SSH, WinRM, etc): 3074.
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Looking for available communicator (SSH, WinRM, etc) port between 222
2 and 4444
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Found available port: 3074 on IP: 127.0.0.1
2023/09/04 08:12:28 ui: ==> qemu.x86-64: Looking for available port between 5900 and 6000 on 127.0.0.1
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Looking for available port between 5900 and 6000 on 127.0.0.1
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Found available port: 5920 on IP: 127.0.0.1
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Found available VNC port: 5920 on IP: 127.0.0.1
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Qemu --version output: QEMU emulator version 7.2.0 (qemu-kvm-7.2.0-14
.el9_2.3)
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers
2023/09/04 08:12:28 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:12:28 Qemu version: 7.2.0
This shows an invocation of the Packer "shell" provisioner to the qemu instance, which is able to connect and run the simple shell inline command
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] scp stderr (length 64): Sink: C0644 46 script_8237.sh
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: debug1: fd 0 clearing O_NONBLOCK
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] Opening new ssh session
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] starting remote command: chmod 0755 /tmp/script_8237.sh
2023/09/04 08:26:14 [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] Opening new ssh session
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] starting remote command: chmod +x /tmp/script_8237.sh; PACKER_BUILDER_TYPE='qemu' PACKER_BUILD_NAME='x86-64' PACKER_HTTP_ADDR='10.0.2.2:8235' PACKER_HTTP_IP='10.0.2.2' PACKER_HTTP_PORT='8235' /tmp/script_8237.sh
2023/09/04 08:26:14 ui: qemu.x86-64: Static hostname: localhost.localdomain
2023/09/04 08:26:14 ui: qemu.x86-64: Transient hostname: localhost
2023/09/04 08:26:14 ui: qemu.x86-64: Icon name: computer-vm
2023/09/04 08:26:14 ui: qemu.x86-64: Chassis: vm 🖴
2023/09/04 08:26:14 ui: qemu.x86-64: Machine ID: 88a379dc76cd4bd4a580e9873cca9fae
2023/09/04 08:26:14 ui: qemu.x86-64: Boot ID: 8808a6183700458292e9239296fbd560
2023/09/04 08:26:14 ui: qemu.x86-64: Virtualization: kvm
2023/09/04 08:26:14 ui: qemu.x86-64: Operating System: Oracle Linux Server 9.2
2023/09/04 08:26:14 ui: qemu.x86-64: CPE OS Name: cpe:/o:oracle:linux:9:2:server
2023/09/04 08:26:14 ui: qemu.x86-64: Kernel: Linux 5.15.0-104.119.4.2.el9uek.x86_64
2023/09/04 08:26:14 ui: qemu.x86-64: Architecture: x86-64
2023/09/04 08:26:14 ui: qemu.x86-64: Hardware Vendor: Red Hat
However, when the ansible provisioner runs, it is connects to the port in the inventory file, which appears to be a connection to the host running the packer command, and not the qemu instance…
Here is the corresponding log showing the ansible provisioner invocation.
2023/09/04 08:26:14 packer-provisioner-shell plugin: [INFO] 0 bytes written for 'stderr'
2023/09/04 08:26:14 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] Opening new ssh session
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] starting remote command: rm -f /tmp/script_8237.sh
2023/09/04 08:26:14 [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] Opening new ssh session
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [DEBUG] starting remote command: rm -f
2023/09/04 08:26:14 [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-plugin-qemu_v1.0.9_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 [INFO] RPC endpoint: Communicator ended with: 0
2023/09/04 08:26:14 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2023/09/04 08:26:14 [INFO] (telemetry) ending shell
2023/09/04 08:26:14 [INFO] (telemetry) Starting provisioner ansible
2023/09/04 08:26:14 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 ansible-playbook version: 2.15.3
2023/09/04 08:26:14 ui: ==> qemu.x86-64: Provisioning with Ansible...
2023/09/04 08:26:14 ui: qemu.x86-64: Setting up proxy adapter for Ansible....
2023/09/04 08:26:14 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 Creating inventory file for Ansible run...
2023/09/04 08:26:14 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/09/04 08:26:14 SSH proxy: serving on 127.0.0.1:34653
If, during the build, I run ansible outside of packer, it will successfully run against the allocated SSH port provided within the above log files.
What is required to make the Ansible provisioner run against the “correct” SSH port , which is dynamically allocated?
Why does the Packer generated inventory file use the (ssh proxy??) port and not the “correct” ssh port?
Thanks in advance for any insights.