Packer 1.9.2 seems that it is unable to find a custom plugin on the current working directory where packer init runs

Hello,

I found that packer version 1.9.2 is unable to find a custom plugin on the current working directory where I run packer init

The following is the error log.

$ ls main.pkr.hcl packer-plugin-docker
main.pkr.hcl		packer-plugin-docker
$
$ packer version
Packer v1.9.2
$ packer init .
$ packer build -only='build.docker.amd64' -except='pp-*' .
Warning: Bundled plugins used

This template relies on the use of plugins bundled into the Packer binary.
The practice of bundling external plugins into Packer will be removed in an
upcoming version.

To remove this warning, add the following section to your template:

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
    docker = {
      source  = "github.com/hashicorp/docker"
      version = "~> 1"
    }
    vagrant = {
      source  = "github.com/hashicorp/vagrant"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

Then run 'packer init' to manage installation of the plugins


build.docker.amd64: output will be in this color.

==> build.docker.amd64: Creating a temporary directory for sharing data...
==> build.docker.amd64: Pulling Docker image: amd64/amazonlinux:2
    build.docker.amd64: 2: Pulling from amd64/amazonlinux
    build.docker.amd64: Digest: sha256:4a3e160c969a06fed62fa091d6cf7e20387b40f330bd0393118cb17bd3abf433
    build.docker.amd64: Status: Image is up to date for amd64/amazonlinux:2
    build.docker.amd64: docker.io/amd64/amazonlinux:2
    build.docker.amd64: What's Next?
    build.docker.amd64: View summary of image vulnerabilities and recommendations → docker scout quickview amd64/amazonlinux:2
==> build.docker.amd64: Starting docker container...
    build.docker.amd64: Run command: docker run --privileged --platform linux/amd64 -v /var/run/secrets/:/var/run/secrets/ -v ~/.aws:/root/.aws -v /Users/{{ username }}/.packer.d/tmp3451986301:/packer-files -d -i -t --entrypoint=/bin/sh -- amd64/amazonlinux:2
==> build.docker.amd64: Error running container: Docker exited with a non-zero exit status.
==> build.docker.amd64: Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
==> build.docker.amd64: See 'docker run --help'.
==> build.docker.amd64:
Build 'build.docker.amd64' errored after 2 seconds 373 milliseconds: Error running container: Docker exited with a non-zero exit status.
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.


==> Wait completed after 2 seconds 373 milliseconds

==> Some builds didn't complete successfully and had errors:
--> build.docker.amd64: Error running container: Docker exited with a non-zero exit status.
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.


==> Builds finished but no artifacts were created.

But if I export the PACKER_PLUGIN_PATH variable, packer build succeeds.

$ export PACKER_PLUGIN_PATH="$PWD"
$ packer build -only='build.docker.amd64' -except='pp-*' .
Warning: Bundled plugins used

This template relies on the use of plugins bundled into the Packer binary.
The practice of bundling external plugins into Packer will be removed in an
upcoming version.

To remove this warning, add the following section to your template:

packer {
  required_plugins {
    docker = {
      source  = "github.com/hashicorp/docker"
      version = "~> 1"
    }
    vagrant = {
      source  = "github.com/hashicorp/vagrant"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
  }
}

Then run 'packer init' to manage installation of the plugins


build.docker.amd64: output will be in this color.

==> build.docker.amd64: Creating a temporary directory for sharing data...
==> build.docker.amd64: Pulling Docker image: amd64/amazonlinux:2
    build.docker.amd64: 2: Pulling from amd64/amazonlinux
    build.docker.amd64: Digest: sha256:4a3e160c969a06fed62fa091d6cf7e20387b40f330bd0393118cb17bd3abf433
    build.docker.amd64: Status: Image is up to date for amd64/amazonlinux:2
    build.docker.amd64: docker.io/amd64/amazonlinux:2
    build.docker.amd64: What's Next?
    build.docker.amd64: View summary of image vulnerabilities and recommendations → docker scout quickview amd64/amazonlinux:2
==> build.docker.amd64: Starting docker container...
    build.docker.amd64: Run command: docker run --privileged --platform linux/amd64 -v /var/run/secrets/:/var/run/secrets/ -v /Users/{{ username }}/.aws:/root/.aws -v /Users/{{ username }}/.packer.d/tmp3353669033:/packer-files -d -i -t --entrypoint=/bin/sh -- amd64/amazonlinux:2
    build.docker.amd64: Container ID: ac6891cf97af1a2125663d31b5c168e108ef46a51f3825256aa120c9428ff820
==> build.docker.amd64: Using docker communicator to connect: 192.168.1.2
==> build.docker.amd64: Provisioning with shell script: files/pre_build.sh
==> build.docker.amd64: + echo y

... ...
... ommitted for simplicity
... ...

==> Builds finished but no artifacts were created.

The plugin document says:

Packer uses the following process for loading the correct plugin:

  1. All directories under the PACKER_PLUGIN_PATH environment variable, if PACKER_PLUGIN_PATH is set. The PACKER_PLUGIN_PATH takes precedences over all other plugin directories; no other directories will be checked.
  2. The directory where packer is installed, or the executable directory.
  3. The current working directory, where packer build is being invoked. (“.”)
  4. The PACKER_CONFIG_DIR/plugins directory. PACKER_CONFIG_DIR refers to Packer’s config directory, if it could be found.

The rule No.3 seems not working.
I would like to hear opinions from the community.
Thanks.

I run the command on MacOS.

Hi @smaruyama this sounds like a bug with the plugin loading logic introduced in version 1.9.2. The warning message as you said indicates that the bundled version of Docker is being loaded and not the custom plugin sitting in the current directory.

Could you enable debug logging to see if it is discover PACKER_LOg=1 packer build main.pkr.hcl?

Locally, I am able to reproduce the issue it finds the plugin but it is reporting it as a non-executable so it is skipped which is odd to me.

2023/07/26 16:27:55 [INFO] Discovered potential plugin: docker = packer-plugin-docker
2023/07/26 16:27:55 [WARN] "packer-plugin-docker" is not executable; skipping

I’ll take a look at this further. If you could check the debug logs for a Discovered statement and any following lines that would be helpful.

Hi, @nywilken
Thank you for your response.
This is the debug log

$ PACKER_LOG=1 packer build -only='build.docker.amd64' -except='pp-*' .
2023/07/27 11:05:08 [INFO] Packer version: 1.9.2 [go1.20.6 darwin amd64]
2023/07/27 11:05:08 [TRACE] discovering plugins in
2023/07/27 11:05:08 [TRACE] discovering plugins in /usr/local/bin
2023/07/27 11:05:08 [INFO] Discovered potential plugin: amazon = /usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.1.0_x5.0_darwin_amd64
2023/07/27 11:05:08 [INFO] Discovered potential plugin: amazon = /usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.2.1_x5.0_darwin_amd64
2023/07/27 11:05:08 [INFO] Discovered potential plugin: vagrant = /usr/local/bin/github.com/hashicorp/vagrant/packer-plugin-vagrant_v1.0.3_x5.0_darwin_amd64
2023/07/27 11:05:08 [INFO] Discovered potential plugin: amazon-ami-management = /usr/local/bin/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_amd64
2023/07/27 11:05:08 [INFO] Discovered potential plugin: amazon-ami-management = /usr/local/bin/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.4.1_x5.0_darwin_amd64
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name-] post-processors from amazon-ami-management plugin
2023/07/27 11:05:08 [INFO] found external [chroot ebs ebssurrogate ebsvolume instance] builders from amazon plugin
2023/07/27 11:05:08 [INFO] found external [import] post-processors from amazon plugin
2023/07/27 11:05:08 found external [ami parameterstore secretsmanager] datasource from amazon plugin
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name-] builders from vagrant plugin
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name- cloud] post-processors from vagrant plugin
2023/07/27 11:05:08 [INFO] Discovered potential plugin: docker = packer-plugin-docker
2023/07/27 11:05:08 [WARN] "packer-plugin-docker" is not executable; skipping
2023/07/27 11:05:08 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/07/27 11:05:08 [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 [WARN] Config file doesn't exist: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 [INFO] Setting cache directory: /Users/{{ username }}/.cache/packer
2023/07/27 11:05:08 Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 [TRACE] listing potential installations for "github.com/wata727/amazon-ami-management" that match ">= 1.3.1". plugingetter.ListInstallationsOptions{FromFolders:[]string{"", "/usr/local/bin"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"darwin", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00016bc80)}}}}
2023/07/27 11:05:08 [TRACE] version "v1.2.0" of file "/usr/local/bin/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.2.0_x5.0_darwin_amd64" does not match constraint ">= 1.3.1"
2023/07/27 11:05:08 [TRACE] Found the following "github.com/wata727/amazon-ami-management" installations: [{/usr/local/bin/github.com/wata727/amazon-ami-management/packer-plugin-amazon-ami-management_v1.4.1_x5.0_darwin_amd64 v1.4.1}]
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name-] post-processors from amazon-ami-management plugin
2023/07/27 11:05:08 [TRACE] listing potential installations for "github.com/hashicorp/vagrant" that match ">= 1.0.3". plugingetter.ListInstallationsOptions{FromFolders:[]string{"", "/usr/local/bin"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"darwin", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00016bc80)}}}}
2023/07/27 11:05:08 [TRACE] Found the following "github.com/hashicorp/vagrant" installations: [{/usr/local/bin/github.com/hashicorp/vagrant/packer-plugin-vagrant_v1.0.3_x5.0_darwin_amd64 v1.0.3}]
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name-] builders from vagrant plugin
2023/07/27 11:05:08 [INFO] found external [-packer-default-plugin-name- cloud] post-processors from vagrant plugin
2023/07/27 11:05:08 [TRACE] listing potential installations for "github.com/hashicorp/amazon" that match ">= 1.1.0". plugingetter.ListInstallationsOptions{FromFolders:[]string{"", "/usr/local/bin"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"darwin", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00016bc80)}}}}
2023/07/27 11:05:08 [TRACE] Found the following "github.com/hashicorp/amazon" installations: [{/usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.1.0_x5.0_darwin_amd64 v1.1.0},{/usr/local/bin/github.com/hashicorp/amazon/packer-plugin-amazon_v1.2.1_x5.0_darwin_amd64 v1.2.1}]
2023/07/27 11:05:08 [INFO] found external [chroot ebs ebssurrogate ebsvolume instance] builders from amazon plugin
2023/07/27 11:05:08 [INFO] found external [import] post-processors from amazon plugin
2023/07/27 11:05:08 found external [ami parameterstore secretsmanager] datasource from amazon plugin
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_playbook_file, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_profile, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_session_token, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_sts_regional_endpoints, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for skip_create_ami, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_secret_key, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_image_amd64, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_push_login_username, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for gosu_version, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ami_name_filter, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for owners, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ami_name, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ami_users, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_tags_to_skip, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ci_commit_ref_name, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for iam_instance_profile, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_clean_staging_directory, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_extra_arguments, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_max_retries, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_web_identity_token_file, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_image_arm64v8, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_box_name, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ami_virtualization_type, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ansible_tags_to_play, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_access_key, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_role_arn, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_tag_alias, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for spot_price, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_source_path, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_s3_upload, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ci_commit_ref_slug, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ci_repository_url, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_push_login_enabled, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for packer_build_dst, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for spot_instance_types, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_skip_package, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_s3_bucket, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_template, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for crz_env_application_level, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for aws_default_region, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ci_project_path, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for docker_push_login_password, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for source_ami_name, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for ssh_username, so skipping
2023/07/27 11:05:08 [TRACE] validateValue: not active for vagrant_box_version, so skipping

This template relies on the use of plugins bundled into the Packer binary.
The practice of bundling external plugins into Packer will be removed in an
upcoming version.

To remove this warning, add the following section to your template:

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
    docker = {
      source  = "github.com/hashicorp/docker"
      version = "~> 1"
    }
    vagrant = {
      source  = "github.com/hashicorp/vagrant"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

Then run 'packer init' to manage installation of the plugins

Warning: Bundled plugins used

This template relies on the use of plugins bundled into the Packer binary.
The practice of bundling external plugins into Packer will be removed in an
upcoming version.

To remove this warning, add the following section to your template:
2023/07/27 11:05:08 [INFO] Starting internal plugin packer-builder-docker

2023/07/27 11:05:08 Starting plugin: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer []string{"/usr/local/Cellar/packer/1.9.2/libexec/bin/packer", "plugin", "packer-builder-docker"}
packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "~> 1"
    }
    docker = {
      source  = "github.com/hashicorp/docker"
      version = "~> 1"
    }
    vagrant = {
      source  = "github.com/hashicorp/vagrant"
      version = "~> 1"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

Then run 'packer init' to manage installation of the plugins


2023/07/27 11:05:08 Waiting for RPC address for: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer
2023/07/27 11:05:08 packer-builder-docker plugin: [INFO] Packer version: 1.9.2 [go1.20.6 darwin amd64]
2023/07/27 11:05:08 packer-builder-docker plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/07/27 11:05:08 packer-builder-docker plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 packer-builder-docker plugin: [WARN] Config file doesn't exist: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 packer-builder-docker plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 packer-builder-docker plugin: [INFO] Setting cache directory: /Users/{{ username }}/.cache/packer
2023/07/27 11:05:08 packer-builder-docker plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 packer-builder-docker plugin: args: []string{"packer-builder-docker"}
2023/07/27 11:05:08 packer-builder-docker plugin: Plugin address: unix /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin1461418361
2023/07/27 11:05:08 packer-builder-docker plugin: Waiting for connection...
2023/07/27 11:05:08 Received unix RPC address for /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: addr is /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin1461418361
2023/07/27 11:05:08 packer-builder-docker plugin: Serving a plugin connection...
2023/07/27 11:05:08 [INFO] Starting internal plugin packer-provisioner-shell
2023/07/27 11:05:08 Starting plugin: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer []string{"/usr/local/Cellar/packer/1.9.2/libexec/bin/packer", "plugin", "packer-provisioner-shell"}
2023/07/27 11:05:08 Waiting for RPC address for: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer
2023/07/27 11:05:08 packer-provisioner-shell plugin: [INFO] Packer version: 1.9.2 [go1.20.6 darwin amd64]
2023/07/27 11:05:08 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/07/27 11:05:08 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:08 packer-provisioner-shell plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 packer-provisioner-shell plugin: [INFO] Setting cache directory: /Users/{{ username }}/.cache/packer
2023/07/27 11:05:08 packer-provisioner-shell plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:08 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2023/07/27 11:05:08 Received unix RPC address for /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: addr is /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin2500975550
2023/07/27 11:05:08 packer-provisioner-shell plugin: Plugin address: unix /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin2500975550
2023/07/27 11:05:08 packer-provisioner-shell plugin: Waiting for connection...
2023/07/27 11:05:08 packer-provisioner-shell plugin: Serving a plugin connection...
2023/07/27 11:05:08 [INFO] Starting internal plugin packer-provisioner-ansible-local
2023/07/27 11:05:08 Starting plugin: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer []string{"/usr/local/Cellar/packer/1.9.2/libexec/bin/packer", "plugin", "packer-provisioner-ansible-local"}
2023/07/27 11:05:08 Waiting for RPC address for: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: [INFO] Packer version: 1.9.2 [go1.20.6 darwin amd64]
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: [WARN] Config file doesn't exist: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: [INFO] Setting cache directory: /Users/{{ username }}/.cache/packer
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: args: []string{"packer-provisioner-ansible-local"}
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: Plugin address: unix /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin3649319876
2023/07/27 11:05:09 Received unix RPC address for /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: addr is /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin3649319876
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: Waiting for connection...
2023/07/27 11:05:09 packer-provisioner-ansible-local plugin: Serving a plugin connection...
2023/07/27 11:05:09 [INFO] Starting internal plugin packer-provisioner-shell
2023/07/27 11:05:09 Starting plugin: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer []string{"/usr/local/Cellar/packer/1.9.2/libexec/bin/packer", "plugin", "packer-provisioner-shell"}
2023/07/27 11:05:09 Waiting for RPC address for: /usr/local/Cellar/packer/1.9.2/libexec/bin/packer
2023/07/27 11:05:09 packer-provisioner-shell plugin: [INFO] Packer version: 1.9.2 [go1.20.6 darwin amd64]
2023/07/27 11:05:09 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/07/27 11:05:09 packer-provisioner-shell plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:09 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /Users/{{ username }}/.packerconfig
2023/07/27 11:05:09 packer-provisioner-shell plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:09 packer-provisioner-shell plugin: [INFO] Setting cache directory: /Users/{{ username }}/.cache/packer
2023/07/27 11:05:09 packer-provisioner-shell plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:09 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2023/07/27 11:05:09 Received unix RPC address for /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: addr is /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin2067408282
2023/07/27 11:05:09 packer-provisioner-shell plugin: Plugin address: unix /var/folders/ly/ycw832yj41n1_zzfngmcrgphgvslj0/T/packer-plugin2067408282
2023/07/27 11:05:09 packer-provisioner-shell plugin: Waiting for connection...
2023/07/27 11:05:09 packer-provisioner-shell plugin: Serving a plugin connection...
build.docker.amd64: output will be in this color.
2023/07/27 11:05:09 Build debug mode: false
2023/07/27 11:05:09 Force build: false

2023/07/27 11:05:09 On error:
2023/07/27 11:05:09 Waiting on builds to complete...
2023/07/27 11:05:09 Starting build run: build.docker.amd64
2023/07/27 11:05:09 Running builder:
2023/07/27 11:05:09 [INFO] (telemetry) Starting builder docker.amd64
2023/07/27 11:05:09 packer-builder-docker plugin: [DEBUG] Docker version: 24.0.0
2023/07/27 11:05:09 packer-builder-docker plugin: [DEBUG] Container will be exported to image.amd64.tar
==> build.docker.amd64: Creating a temporary directory for sharing data...
2023/07/27 11:05:09 packer-builder-docker plugin: Old default config directory found: /Users/{{ username }}/.packer.d
2023/07/27 11:05:09 packer-builder-docker plugin: Set Packer temp dir to /Users/{{ username }}/.packer.d/tmp850088789
==> build.docker.amd64: Pulling Docker image: amd64/amazonlinux:2
2023/07/27 11:05:09 packer-builder-docker plugin: Executing: /usr/local/bin/docker [pull amd64/amazonlinux:2 --platform linux/amd64]
    build.docker.amd64: 2: Pulling from amd64/amazonlinux
    build.docker.amd64: a8d554425610: Pulling fs layer
    build.docker.amd64: a8d554425610: Verifying Checksum
    build.docker.amd64: a8d554425610: Download complete
    build.docker.amd64: a8d554425610: Pull complete
    build.docker.amd64: Digest: sha256:8493322bcbf25417bb2acf5de53302811c30558965fb9c91bd8dfe3a9db7a06e
    build.docker.amd64: Status: Downloaded newer image for amd64/amazonlinux:2
    build.docker.amd64: docker.io/amd64/amazonlinux:2
    build.docker.amd64: What's Next?
    build.docker.amd64: View summary of image vulnerabilities and recommendations → docker scout quickview amd64/amazonlinux:2
==> build.docker.amd64: Starting docker container...
    build.docker.amd64: Run command: docker run --privileged --platform linux/amd64 -v /var/run/secrets/:/var/run/secrets/ -v ~/.aws:/root/.aws -v /Users/{{ username }}/.packer.d/tmp850088789:/packer-files -d -i -t --entrypoint=/bin/sh -- amd64/amazonlinux:2
2023/07/27 11:05:33 packer-builder-docker plugin: Starting container with args: [run --privileged --platform linux/amd64 -v /var/run/secrets/:/var/run/secrets/ -v ~/.aws:/root/.aws -v /Users/{{ username }}/.packer.d/tmp850088789:/packer-files -d -i -t --entrypoint=/bin/sh -- amd64/amazonlinux:2]
2023/07/27 11:05:33 packer-builder-docker plugin: Waiting for container to finish starting
==> build.docker.amd64: Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
==> build.docker.amd64: See 'docker run --help'.
==> build.docker.amd64:
==> build.docker.amd64: Error running container: Docker exited with a non-zero exit status.
==> build.docker.amd64: Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
==> build.docker.amd64: See 'docker run --help'.
==> build.docker.amd64:
2023/07/27 11:05:34 [INFO] (telemetry) ending docker.amd64
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.

==> Wait completed after 25 seconds 124 milliseconds
Build 'build.docker.amd64' errored after 25 seconds 124 milliseconds: Error running container: Docker exited with a non-zero exit status.
2023/07/27 11:05:34 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/07/27 11:05:34 machine readable: build.docker.amd64,error []string{"Error running container: Docker exited with a non-zero exit status.\nStderr: docker: Error response from daemon: create ~/.aws: \"~/.aws\" includes invalid characters for a local volume name, only \"[a-zA-Z0-9][a-zA-Z0-9_.-]\" are allowed. If you intended to pass a host directory, use absolute path.\nSee 'docker run --help'.\n"}
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.
==> Builds finished but no artifacts were created.
2023/07/27 11:05:34 [INFO] (telemetry) Finalizing.
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.


==> Wait completed after 25 seconds 124 milliseconds

==> Some builds didn't complete successfully and had errors:
--> build.docker.amd64: Error running container: Docker exited with a non-zero exit status.
Stderr: docker: Error response from daemon: create ~/.aws: "~/.aws" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.


==> Builds finished but no artifacts were created.
2023/07/27 11:05:34 waiting for all plugin processes to complete...
2023/07/27 11:05:34 /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: plugin process exited
2023/07/27 11:05:34 /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: plugin process exited
2023/07/27 11:05:34 /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: plugin process exited
2023/07/27 11:05:34 /usr/local/Cellar/packer/1.9.2/libexec/bin/packer: plugin process exited

Found the same WARN message.

			if _, err := exec.LookPath(pluginPath); err != nil {
				log.Printf("[WARN] %q is not executable; skipping", pluginPath)
				continue
			}

exec.LookPath seems unable to find the custom plugin.

Thanks for taking a look at the post.

@smaruyama thanks for the reporting back your findings and confirming the behavior. I opened an issue on GitHub to track this regression Packer 1.9.2 is unable to find a custom plugin on the current working directory · Issue #12543 · hashicorp/packer · GitHub

It looks with the introduction of exec.LookPath to validate that the plugin is executable failed to load the plugin because of a security change made within Go 1.19. The additional call to exec.LookPath did not exist prior to 1.9.2 so this was not an issue because the plugin execution call was always using the absolute path.

I opened a PR with a potential fix as well.

@nywilken Thanks for your response and opening the potential fix PR.