Shell provisioner: How to pass env var containing $ (dollar sign)

Hello!

Summary

I would like to create a provisioner that can pull a docker image from a private repository.

I already have a private repo on the free Dockerhub which I am able to push/pull after I login.

The following works:

# From my shell on macOS:
vagrant ssh

Inside the VM:

# From my Ubuntu VM hashicorp/bionic64
docker login -u my_username -p 'my_pas$word' # Note that I have single-quoted my password because it contains a $ (dollar character).
docker pull my_username/my_repo

The following does NOT work:

I’m having problems with the following snippet from my Vagrantfile:

  config.vm.provision "docker"

  config.vm.provision "docker-pull", type: "shell" do |s|
    s.env = { 'DOCKERHUB_USERNAME' => ENV['DOCKERHUB_USERNAME'],
              'DOCKERHUB_PASSWORD' => ENV['DOCKERHUB_PASSWORD'] }

    s.inline = "docker login -u $DOCKERHUB_USERNAME -p $DOCKERHUB_PASSWORD && docker pull my_username/my_repo"
  end

The docker login command fails because the password contains a $ which is interpreted by the shell as the start of a variable name.

Details

Here is a simple test to illustrate my problem:

  config.vm.provision "dollar", type: "shell" do |s|
    s.env = { 'DOCKERHUB_USERNAME' => ENV['DOCKERHUB_USERNAME'],
              'DOCKERHUB_PASSWORD' => ENV['DOCKERHUB_PASSWORD'] }
    puts s.env
    s.inline = 'env | grep DOCKERHUB_'
  end

and here is my command and its output:

env | grep DOCKERHUB_ && vagrant provision --provision-with dollar
DOCKERHUB_USERNAME=username
DOCKERHUB_PASSWORD=Pas$word
{"DOCKERHUB_USERNAME"=>"username", "DOCKERHUB_PASSWORD"=>"Pas$word"}
==> seals-local: Running provisioner: dollar (shell)...
    seals-local: Running: inline script
    seals-local: DOCKERHUB_PASSWORD=Pas
    seals-local: DOCKERHUB_USERNAME=username

So,

  1. On the host, DOCKERHUB_PASSWORD is correctly stored and displayed.
  2. Inside the Vagrantfile, s.env is correclty stored and displayed.
  3. However, inside the VM, the password is truncated at the $ sign.

Debugging

vagrant --debug provision --provision-with dollar

I see the following in the debug output:

 INFO ssh: Execute: chmod +x '/tmp/vagrant-shell' && DOCKERHUB_USERNAME="username" DOCKERHUB_PASSWORD="Pas$word" /tmp/vagrant-shell (sudo=true)

I do:

vagrant ssh

and inside the VM:

vagrant@linux:~$ DOCKERHUB_USERNAME="username" DOCKERHUB_PASSWORD="Pas$word" /tmp/vagrant-shell
DOCKERHUB_PASSWORD=Pas
DOCKERHUB_USERNAME=username
vagrant@linux:~$

I change the double-quotes to single quotes:

vagrant@linux:~$ DOCKERHUB_USERNAME='username' DOCKERHUB_PASSWORD='Pas$word' /tmp/vagrant-shell
DOCKERHUB_PASSWORD=Pas$word
DOCKERHUB_USERNAME=username
vagrant@linux:~$

and now it works.

So, I need a way to tell Vagrant to use single-quotes instead of double-quotes when passing environment variables to the VM.

SSH Settings

In the Vagrant manual, there seems to be an SSH setting which is described as follows:

config.ssh.export_command_template (string) - The template used to generate exported environment variables in the active session. This can be useful when using a Bourne incompatible shell like C shell. The template supports two variables which are replaced with the desired environment variable key and environment variable value: %ENV_KEY% and %ENV_VALUE% . The default template is:

config.ssh.export_command_template = 'export %ENV_KEY%="%ENV_VALUE%"'

However, I think this only applies to the vagrant ssh command but not to the vagrant provision command, since the logs did not have the ‘export ’ but simply placed VAR="VALUE" in front of /tmp/vagrant-shell. I confirmed this in the source code:

# vagrant/plugins/provisioners/shell/provisioner.rb

      # This is the provision method called if SSH is what is running
      # on the remote end, which assumes a POSIX-style host.
      def provision_ssh(args)
        env = config.env.map { |k,v| "#{k}=#{quote_and_escape(v.to_s)}" }
        env = env.join(" ")

        command =  "chmod +x '#{upload_path}'"
        command << " &&"
        command << " #{env}" if !env.empty?
        command << " #{upload_path}#{args}"

So, the #{env} is added after the chmod +x /tmp/vagrant-shell and it is supposed to have been processed with quote_and_escape but this method:

      # Quote and escape strings for shell execution, thanks to Capistrano.
      def quote_and_escape(text, quote = '"')
        "#{quote}#{text.gsub(/#{quote}/) { |m| "#{m}\\#{m}#{m}" }}#{quote}"
      end

seems to use the default quote as a double-quote, and there doesn’t seem to be a way for me to tell this method to use single-quotes or to escape the $ sign.

I would have liked to be able to do something like:

config.provision.export_command_template = "%ENV_KEY%='%ENV_VALUE%'" # Note the single-quotes

Note the single-quotes around %ENV_VALUE%.

Looking at the shell provisioner documentation, it says:

env (hash) - List of key-value pairs to pass in as environment variables to the script. Vagrant will handle quoting for environment variable values, but the keys remain untouched.

Maybe I’m missing something about how quoting works?

Insecure workaround

I did find an insecure work-around as follows:

  config.vm.provision "docker-pull", type: "shell" do |s|
    username = ENV['DOCKERHUB_USERNAME']
    password = ENV['DOCKERHUB_PASSWORD']
    s.inline = "echo '#{password}' | docker login -u '#{username}' --password-stdin && docker pull #{username}/repo"
  end

However, this method will show the password in the shell history on the VM and inside /tmp/vagrant-shell

And, yes, I can change my password to not contain a $, but this doesn’t seem like the best solution to the problem. What if a password contains another special character that causes another problem with shell quoting?

I would prefer to find the correct solution, or maybe this is a bug (in which case I would create a Github issue).

Please help!

A bit more info from the log:

 INFO global: Vagrant version: 2.2.7
 INFO global: Ruby version: 2.4.9
 INFO global: RubyGems version: 2.6.14.4
 INFO global: VAGRANT_LOG="info"
 INFO global: VAGRANT_INSTALLER_VERSION="2"
 INFO global: VAGRANT_EXECUTABLE="/opt/vagrant/embedded/gems/2.2.7/gems/vagrant-2.2.7/bin/vagrant"
 INFO global: VAGRANT_INSTALLER_EMBEDDED_DIR="/opt/vagrant/embedded"
 INFO global: VAGRANT_INSTALLER_ENV="1"

I think this line:

env = config.env.map { |k,v| "#{k}=#{quote_and_escape(v.to_s)}" }

should be changed to:

env = config.env.map { |k,v| "#{k}=#{quote_and_escape(v.to_s, "'")}" }

This way, any values of environment variables will not be subject to variable expansion by the shell.

After I did this change locally on my machine, here is the debug output:

 INFO ssh: Execute: chmod +x '/tmp/vagrant-shell' && DOCKERHUB_USERNAME='username' DOCKERHUB_PASSWORD='Pas$word' /tmp/vagrant-shell (sudo=true)

and here is the regular output:

DOCKERHUB_USERNAME=username
DOCKERHUB_PASSWORD=Pas$word
{"DOCKERHUB_USERNAME"=>"username", "DOCKERHUB_PASSWORD"=>"Pas$word"}
You appear to be running Vagrant outside of the official installers.
Note that the installers are what ensure that Vagrant has all required
dependencies, and Vagrant assumes that these dependencies exist. By
running outside of the installer environment, Vagrant may not function
properly. To remove this warning, install Vagrant using one of the
official packages from vagrantup.com.

==> default: Running provisioner: dollar (shell)...
    default: Running: inline script
    default: DOCKERHUB_PASSWORD=Pas$word
    default: DOCKERHUB_USERNAME=username