Vsphere-clone with Ubuntu OVAs?

Packer
1

(reposted from here)

  • Packer v1.7.1
  • vCenter Server 6.7

Having a helluva time getting this working, specifically the ssh connection post-build. Packer will clone & boot the OVA ok, but then errors “waiting for SSH to become available.”

Using the official bionic & focal OVAs, which come with the following params/properties:

bash-5.1# ovftool bionic-server-cloudimg-amd64.ova 
OVF version:   1.0
VirtualApp:    false
Name:          Ubuntu 18.04 Server (20210609)

Download Size:  337.39 MB

Deployment Sizes:
  Flat disks:   10.00 GB
  Sparse disks: Unknown

Networks:
  Name:        VM Network
  Description: The VM Network network

Virtual Machines:
  Name:               ubuntu-bionic-18.04-cloudimg-20210609
  Operating System:   ubuntu64guest
  Virtual Hardware:
    Families:         vmx-10 
    Number of CPUs:   2
    Cores per socket: 1
    Memory:           1024.00 MB

    Disks:
      Index:          0
      Instance ID:    9
      Capacity:       10.00 GB
      Disk Types:     SCSI-VirtualSCSI 

    NICs:
      Adapter Type:   VmxNet3
      Connection:     VM Network

Properties:
  Key:         instance-id
  Label:       A Unique Instance ID for this instance
  Type:        string
  Description: Specifies the instance id.  This is required and used to 
               determine if the machine should take "first boot" actions 
  Value:       id-ovf

  Key:         hostname
  Type:        string
  Description: Specifies the hostname for the appliance
  Value:       ubuntuguest

  Key:         seedfrom
  Label:       Url to seed instance data from
  Type:        string
  Description: This field is optional, but indicates that the instance should 
               'seed' user-data and meta-data from the given url.  If set to 
               'http://example.com/sm-' is given, meta-data will be pulled from
               http://example.com/sm-meta-data and user-data from 
               http://example.com/sm-user-data.  Leave this empty if you do not
               want to seed from a url. 

  Key:         public-keys
  Label:       ssh public keys
  Type:        string
  Description: This field is optional, but indicates that the instance should 
               populate the default user's 'authorized_keys' with this value 

  Key:         user-data
  Label:       Encoded user-data
  Type:        string
  Description: In order to fit into a xml attribute, this value is base64 
               encoded . It will be decoded, and then processed normally as 
               user-data. 

  Key:         password
  Label:       Default User's password
  Type:        string
  Description: If set, the default user's password will be set to this value to
               allow password based login.  The password will be good for only 
               a single login.  If set to the string 'RANDOM' then a random 
               password will be generated, and written to the console. 

References:
  File:  ubuntu-bionic-18.04-cloudimg.vmdk

So:

  • instance-id
  • hostname
  • seedfrom
  • public-keys
  • user-data
  • password

I’m passing some of these properties via Packer’s vsphere-clone builder; here’s a snip of my template file:

source "vsphere-clone" "ubuntu-18" {
  template            = "ubuntu-18.ova"
  network             = "${var.networkName}"
  CPUs                = "${var.cores}"
  RAM                 = "${var.memMB}"
  boot_wait           = "5s"
  communicator        = "ssh"
  create_snapshot     = true
  datacenter          = "${var.vcenter_dc}"
  datastore           = "${var.vcenter_datastore}"
  cluster             = "${var.vcenter_cluster}"
  folder              = "${var.vcenter_template_folder}"
  convert_to_template = true
  notes                          = "${var.vm_notes}"
  password                       = "${var.vcenter_pass}"
  shutdown_command               = "sudo shutdown -P now"
  ssh_handshake_attempts         = "50"
  ssh_private_key_file           = "${var.privateKey}"
  ssh_pty                        = true
  ssh_timeout                    = "20m"
  ssh_username                   = "admin"
  username       = "${var.vcenter_user}"
  vcenter_server = "${var.vcenter_server}"
  insecure_connection = true
  vm_name        = "${var.vm_name}"
  vapp {
    properties = {
      instance-id  = "${var.vm_name}"
      hostname     = "${var.vm_name}"
      public-keys  = "${var.publicKey}"
      user-data    = "${var.userData}"
    }
  }
}

Oddly, the instance-id and hostname properties are being set just fine on boot. Here is my user-data, I’m passing the same public key from the below user-data to the “public-keys” property above:

#cloud-config
chpasswd:
  list: |
    ubuntu:<generic pass>
  expire: False
users:
  - name: admin
    gecos: admin
    ssh_authorized_keys:
      - <pubkey>
    groups: sudo, users, admin
    shell: /bin/bash
    sudo: ["ALL=(ALL:ALL) NOPASSWD:ALL"]
disable_root: true
package_upgrade: true
packages:
  - open-vm-tools

So the flow is: pass user-data from vapp properties > creates admin user with allowed key > private key used to ssh auth. But after it boots to the prompt, it errors with: “Packer experienced an authentication error when trying to connect via SSH.” I started the day today with the complete opposite setup, where there were no keys involved at all; using passwords-only, with the same result. I wasn’t able to log in on the console with either the default user or my created account using the default & my created passwords.

Once the instance is booted, Packer will wait ~6mins before giving up. In that time, I’ve tried to ssh to the instance manually, using the same private key & both the default (“ubuntu”) & my created “admin” account. I get the, “Permission denied (publickey)” error immediately.

Command used for building is as follows:

packer build -only "*.ubuntu-20" -var vm_name=ubuntu-20-test -var-file <contains vcenter vars> templates/ubuntu.pkr.hcl

Where the vcenter password and ssh_private_key_file values exist as environment variables.