Packer with Provisioners Crashes Everytime

Hi all,

I am attempting to add an ansible local provisioner to the MAAS Cent OS 7 packer build file and it crashes every time with a “Build ‘qemu’ errored: unexpected EOF” and “panic: runtime error: invalid memory address or nil pointer dereference” error.

I have tried plain shell commands as the provisioner and I still get the same error. The qemu builder will get to the Provisioning section, try whatever Provisioner I am using (I have tried Ansible remote & local, and shell), and will fail immediately. I can’t even use the VNC because it crashes so quickly. If I remove the provisioners, the install work. I can even use the VNC. What am I doing wrong if I may ask? I have listed the build machine specs and packer version below. Thanks for the help in advance.

Build Machine
Dell Precision T3610: Intel® Xeon® E5-1603 Processor & 32 GB Ram

Packer Version
1.5.4

Modified Build File
{
“provisioners”: [
{
“type”: “shell”,
“inline”: [“sudo yum install epel-release”,“sudo yum install ansible”]
},
{
“type”: “ansible-local”,
“playbook_file”: “./playbook.yml”,
“role_paths”: ["./roles"]
}
],
“builders”: [
{
“type”: “qemu”,
“communicator”: “none”,
“iso_url”: “https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-NetInstall-1908.iso”,
“iso_checksum_type”: “sha256”,
“iso_checksum_url”: “https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/sha256sum.txt”,
“boot_command”: [
" ",
"inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos7.ks ",
“”
],
“boot_wait”: “3s”,
“disk_size”: “4G”,
“headless”: true,
“memory”: 1024,
“http_directory”: “http”,
“shutdown_timeout”: “20m”
}
],
“post-processors”: [
{
“type”: “shell-local”,
“inline_shebang”: “/bin/bash -e”,
“inline”: [
“TMP_DIR=$(mktemp -d /tmp/packer-maas-XXXX)”,
“echo ‘Mounting image…’”,
“modprobe nbd”,
“qemu-nbd -d /dev/nbd4”,
“qemu-nbd -c /dev/nbd4 -n output-qemu/packer-qemu”,
“echo ‘Waiting for partitions to be created…’”,
“tries=0”,
“while [ ! -e /dev/nbd4p1 -a tries -lt 60 ]; do", " sleep 1", " tries=((tries+1))”,
“done”,
“echo ‘Tarring up image…’”,
“mount /dev/nbd4p1 $TMP_DIR”,
“tar -czpf centos7.tar.gz -C $TMP_DIR .”,
“echo ‘Unmounting image…’”,
“umount $TMP_DIR”,
“qemu-nbd -d /dev/nbd4”,
“rmdir $TMP_DIR”
]
}
]
}

Error
==> qemu: Connecting to VM via VNC (127.0.0.1:5980)
==> qemu: Typing the boot command over VNC…
==> qemu: Provisioning with shell script: /tmp/packer-shell750892231
Build ‘qemu’ errored: unexpected EOF

==> Some builds didn’t complete successfully and had errors:
–> qemu: unexpected EOF

==> Builds finished but no artifacts were created.
panic: runtime error: invalid memory address or nil pointer dereference
2020/03/06 10:12:56 packer-builder-qemu plugin: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xf8da7d]
2020/03/06 10:12:56 packer-builder-qemu plugin:
2020/03/06 10:12:56 packer-builder-qemu plugin: goroutine 170 [running]:
2020/03/06 10:12:56 packer-builder-qemu plugin: github.com/hashicorp/packer/packer/rpc.(*CommunicatorServer).Upload(0xc0002d4990, 0xc00059a420, 0xc000414880, 0x0, 0x0)
2020/03/06 10:12:56 packer-builder-qemu plugin: /Users/mmarsh/Projects/packer/packer/rpc/communicator.go:301 +0xfd
2020/03/06 10:12:56 packer-builder-qemu plugin: reflect.Value.call(0xc0004ccd20, 0xc0004f2188, 0x13, 0x422ae46, 0x4, 0xc00074af18, 0x3, 0x3, 0x610a534146764472, 0x4f4c616b574f6764, …)
2020/03/06 10:12:56 packer-builder-qemu plugin: /usr/local/go/src/reflect/value.go:460 +0x5f6
2020/03/06 10:12:56 packer-builder-qemu plugin: reflect.Value.Call(0xc0004ccd20, 0xc0004f2188, 0x13, 0xc000620f18, 0x3, 0x3, 0x504144517749306f, 0x424d5248564e6742, 0x4441544245386641)
2020/03/06 10:12:56 packer-builder-qemu plugin: /usr/local/go/src/reflect/value.go:321 +0xb4
2020/03/06 10:12:56 packer-builder-qemu plugin: net/rpc.(*service).call(0xc00041e280, 0xc00056e7d0, 0xc00065b358, 0xc00065b360, 0xc000428e00, 0xc00059a3a0, 0x3468e20, 0xc00059a420, 0x16, 0x34e18e0, …)
2020/03/06 10:12:56 packer-builder-qemu plugin: /usr/local/go/src/net/rpc/server.go:377 +0x16f
2020/03/06 10:12:56 packer-builder-qemu plugin: created by net/rpc.(*Server).ServeCodec
2020/03/06 10:12:56 packer-builder-qemu plugin: /usr/local/go/src/net/rpc/server.go:474 +0x42b
2020/03/06 10:12:56 /usr/bin/packer: plugin process exited
2020/03/06 10:12:56 [INFO] (telemetry) ending qemu
2020/03/06 10:12:56 ui error: Build ‘qemu’ errored: unexpected EOF
2020/03/06 10:12:56 machine readable: error-count string{“1”}
2020/03/06 10:12:56 ui error:
==> Some builds didn’t complete successfully and had errors:
2020/03/06 10:12:56 machine readable: qemu,error string{“unexpected EOF”}
2020/03/06 10:12:56 ui error: --> qemu: unexpected EOF
2020/03/06 10:12:56 ui:
==> Builds finished but no artifacts were created.
2020/03/06 10:12:56 [INFO] (telemetry) Finalizing.
2020/03/06 10:12:56 packer-provisioner-shell plugin: Retryable error: Error uploading script: unexpected EOF
2020/03/06 10:12:56 waiting for all plugin processes to complete…
2020/03/06 10:12:56 /usr/bin/packer: plugin process exited
2020/03/06 10:12:56 [INFO] (telemetry) ending shell
2020/03/06 10:12:56 /usr/bin/packer: plugin process exited
2020/03/06 10:12:56 /usr/bin/packer: plugin process exited

!!! PACKER CRASH !!!

Packer crashed! This is always indicative of a bug within Packer.
A crash log has been placed at “crash.log” relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Packer1 so that we can fix this.

There is something wrong - maybe a missing closing bracket or apostrophe - with your post-processor script. Can you format the whole packer script so that we can debug it? Try using a code block.

Sure thing, sorry about that.

{
  "provisioners": [
    {
      "type": "shell",
      "inline": ["sudo yum install epel-release","sudo yum install ansible"]
    },
    {
      "type": "ansible-local",
      "playbook_file": "./playbook.yml",
      "role_paths": ["./roles"]
    }
  ],
  "builders": [
    {
      "type": "qemu",
      "communicator": "none",
      "iso_url": "https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-NetInstall-1908.iso",
      "iso_checksum_type": "sha256",
      "iso_checksum_url": "https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/sha256sum.txt",
      "boot_command": [
        "<tab> ",
        "inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos7.ks ",
        "<enter>"
      ],
      "boot_wait": "3s",
      "disk_size": "4G",
      "headless": true,
      "memory": 2048,
      "http_directory": "http",
      "shutdown_timeout": "20m"
    }
  ],
  "post-processors": [
    {
      "type": "shell-local",
      "inline_shebang": "/bin/bash -e",
      "inline": [
        "TMP_DIR=$(mktemp -d /tmp/packer-maas-XXXX)",
        "echo 'Mounting image...'",
        "modprobe nbd",
        "qemu-nbd -d /dev/nbd4",
        "qemu-nbd -c /dev/nbd4 -n output-qemu/packer-qemu",
        "echo 'Waiting for partitions to be created...'",
        "tries=0",
        "while [ ! -e /dev/nbd4p1 -a $tries -lt 60 ]; do",
        "    sleep 1",
        "    tries=$((tries+1))",
        "done",
        "echo 'Tarring up image...'",
        "mount /dev/nbd4p1 $TMP_DIR",
        "tar -czpf centos7.tar.gz -C $TMP_DIR .",
        "echo 'Unmounting image...'",
        "umount $TMP_DIR",
        "qemu-nbd -d /dev/nbd4",
        "rmdir $TMP_DIR"
      ]
    }
  ]
}

First you could try sudo yum install -y [...] for answering the yum questions automatically with yes. I don’t know on the fly if Packer is answering the questions by itself.

For the rest of the script I’ll need some more time. :blush:

Good catch, Wolf. I will update that piece of code. Take all the time you need, I really appreciate your help. :+1: :+1: :+1:

I think I will need the playbook, too. Just to be safe with that.

Ok, no problem. I attached it. It is for active directory so I had to remove all of the sensitive data and combine the role and the playbook since I can’t upload the files using the uploader on this form.

---
- name: Configure Active Directory Authentication
  hosts: localhost
  vars:
  tasks:
  - name: Install Packages - Sssd, sssd-client, krb5-workstation, samba, openldap-clients, policycoreutils-python, oddjob, oddjob-mkhomedir
    yum:
      name:
        - sssd
        - sssd-client
        - krb5-workstation
        - samba
        - openldap-clients
        - policycoreutils-python
        - oddjob
        - oddjob-mkhomedir
      state: present
      update_cache: yes

  - name: Running authconfig
    command: authconfig --enablesssd --enablesssdauth --enablemkhomedir --update
    notify:
      - restart sssd

  - name: Add sssd.conf
    template:
      src: sssd.conf.j2
      dest: /etc/sssd/sssd.conf
      owner: root
      group: root
      mode: '0600'

  - name: Add smb.conf
    copy:
      src: files/smb.conf
      dest: /etc/samba/smb.conf
      owner: root
      group: root
      mode: '0644'

  - name: Add krb5.conf
    copy:
      src: files/krb5.conf
      dest: /etc/krb5.conf
      owner: root
      group: root
      mode: '0644'

  - name: Add oddjobd-mkhomedir.conf
    copy:
      src: files/oddjobd-mkhomedir.conf
      dest: /etc/oddjobd.conf.d/oddjobd-mkhomedir.conf
      owner: root
      group: root
      mode: '0400'
    notify:
      - restart oddjobd

  - name: Add 5VC-4DJ01N.keytab
    copy:
      src: files/5VC-4DJ01N.keytab
      dest: /root/5VC-4DJ01N.keytab
      owner: root
      group: root
      mode: '0400'

  - name: Adjust file context mapping for krb5.conf
    command: restorecon /etc/krb5.conf

  - name: Adjust file context mapping for sssd.conf
    command: restorecon /etc/sssd/sssd.conf

  - name: Adjust file context mapping for smb.conf
    command: restorecon /etc/samba/smb.conf

  - name: Authenticate and create ticket 3/3
    command: /usr/bin/net ads join -k createcomputer="Org/_Computers"

  - name: Adjust file context mapping for sssd.conf
    command: restorecon /etc/sssd/sssd.conf

  - name: Run authconfig
    command: authconfig --enablesssd --enablesssdauth --enablemkhomedir --update
    notify:
      - restart sssd
packer build packer.json
qemu: output will be in this color.

==> qemu: Retrieving ISO
==> qemu: Trying https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-NetInstall-1908.iso
==> qemu: Trying https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-NetInstall-1908.iso?checksum=sha256%3A6ffa7ad44e8716e4cd6a5c3a85ba5675a935fc0448c260f43b12311356ba85ad
==> qemu: https://mirrors.edge.kernel.org/centos/7.7.1908/isos/x86_64/CentOS-7-x86_64-NetInstall-1908.iso?checksum=sha256%3A6ffa7ad44e8716e4cd6a5c3a85ba5675a935fc0448c260f43b12311356ba85ad => /home/
twolf/workspace/tmp/packer_cache/a25429a701da3231519f3a2aea0bb6b76220f38b.iso
==> qemu: Creating required virtual machine disks
==> qemu: Starting HTTP server on port 8371
==> qemu: Looking for available port between 5900 and 6000 on 127.0.0.1
==> qemu: Starting VM, booting from CD-ROM
    qemu: The VM will be run headless, without a GUI. If you want to
    qemu: view the screen of the VM, connect via VNC without a password to
    qemu: vnc://127.0.0.1:5976
==> qemu: Waiting 3s for boot...
==> qemu: Connecting to VM via VNC (127.0.0.1:5976)
==> qemu: Typing the boot command over VNC...
==> qemu: Provisioning with shell script: /tmp/packer-shell358150562
Build 'qemu' errored: unexpected EOF

==> Some builds didn't complete successfully and had errors:
--> qemu: unexpected EOF

==> Builds finished but no artifacts were created.

But that’s correct, because i’m missing the centos7.ks file. :slight_smile: Either is your error in the kickstart file or maybe the file isn’t at the right place where packer expects it. The port to upload the file should be open, too.

As your debug log tries to tell you, i guess (i’ve overseen this, too):

[...]
2020/03/06 10:12:56 packer-provisioner-shell plugin: Retryable error: Error uploading script: unexpected EOF
[...]

Thanks for the reply and help, Wolf.

I kept the kickstart file in the original place the MAAS repo has it which is in the http folder. The build works with no provisioners, but not with them. I even copied the ks file to the root and it still fails.

I am going to attempt a double build to see if it works. Thanks again for the help!

I’m having a similar issue, did you ever find a resolution? Our scripts work against VMWare but not qemu