Download box from authenticated repo

I have base images hosted in JFrog’s Artifactory.

In order to manually download them I run these commands

export ATLAS_TOKEN=$(curl --silent --user my_user https://example.com/artifactory/api/vagrant/auth)
export VAGRANT_SERVER_URL=http://example.com/artifactory/api/vagrant/vagrant-local
export VAGRANT_SERVER_ACCESS_TOKEN_BY_URL=1
vagrant box add "https://example.com/artifactory/api/vagrant/vagrant-local/my_box.box" --name my-box

A colleague came up with this Ruby snippet for the Vagrantfile

if ['up', 'reload'].include?(ARGV[0])
  #
  # Artifactory ATLAS _TOKEN handling
  #
  require 'net/http'
  require 'uri'
  require 'io/console'

  unless ENV["USERNAME"].nil?
    username = ENV["USERNAME"]
  else
    puts "Enter your username:  "
    username = STDIN.gets.chomp
  end
  password = STDIN.getpass("Password:")

  uri = URI.parse($vagrant_server_base_url + "auth")
  request = Net::HTTP::Get.new(uri)
  request.basic_auth(username, password)

  req_options = {
    use_ssl: uri.scheme == "https",
  }

  response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
    http.request(request)
  end

  puts 'Response code from Artifactory for password retrieval: ' + response.code

  if response.code.to_i == 200
    ENV["ATLAS_TOKEN"] = response.body
    # puts ENV["ATLAS_TOKEN"]
  else 
    puts "Can't obtain ATLAS_TOKEN from Artifactory."
    abort ('Aborting ...' )
  end

  # https://www.jfrog.com/confluence/display/JFROG/Vagrant+Repositories
  ENV["VAGRANT_SERVER_URL"] = $vagrant_repo_url
  # https://www.vagrantup.com/docs/other/environmental-variables#vagrant_server_access_token_by_url
  ENV["VAGRANT_SERVER_ACCESS_TOKEN_BY_URL"] = "1"
end

During our tests we found and filed this bug #12402.

Our intention is to make an authentication plugin out of this.

But first we’d need some guidance as to how to embed that code in a way that seamlessly works with other checks performed by Vagrant, e.g., when you delete a box, when it checks for updates, etc.

How would you go about it?

Hi @soapy1

You were of huge help with the none communicator.

Could you give us some guidance with this code?

We would like to write a plugin for authenticating to repos.

Hmmm, so there exists some documentation for this.

Talks about how to setup a plugin.

This blog post is a bit more in depth

I’m guessing the easiest way to get this to run at the right time is to use a hook. Plugin Development Basics - Action Hooks | Vagrant by HashiCorp

Another option, if you already have code to do what you want in the form of a Vagrantfile is to rely on Vagrantfile merging and drop that snipped into a Vagrantfile in a users ~/vagrant.d directory.

I hope this gives you some direction

1 Like

Hi @soapy1, these days we are resuming this side project. Is there any way to get a list of the vms to be spawned by Vagrant upon running vagrant up?

Edit: I think I found it. We basically want to pass the auth plugin a list of vms to download. We are going to try to diff machine_names and BoxCollection#all, but I’d appreciate your insight.

As I understand machine_names is a method of Vagrantfile class
In order to have an instance of VagrantFile class, we need to initialize . Initialize has two input parameters…

Initializes by loading a Vagrantfile.

Parameters:

loader (Config::Loader) — Configuration loader that should already be configured with the proper Vagrantfile locations. This usually comes from Environment
keys (Array<Symbol>) — The Vagrantfiles to load and the order to load them in (keys within the loader).

We have Config::Loader in environment. However we also need keys (Array) . I couldn’t understand where we can get this keys array.

Thank you.
Alperen

Hi @soapy1, could you please give us a hint here?

Hey there,

the keys (Array) is the list of keys to load from the config loader. You choose as the user of this module what configs to load.

So, when you setup the config loader there are these sources which are named paths that define where a Vagrantfile is. So, you can have multiple Vagrantfiles that get merged together. Practically, in Vagrant these paths come from the box, the root environment or the project, etc. You can set new sources in the config loader to define what files need to be merged together to get a final config.

Say you have one Vagrantfile that you want to load then that can look something like:

# Setup the config loader
config_loader = Vagrant::Config::Loader.new(
  Vagrant::Config::VERSIONS, Vagrant::Config::VERSIONS_ORDER)

# Add the path to my_vagrantfile to the config loader
config_loader.set(:root, "/path/to/my_vagrantfile")

# Create the Vagrantfile object. In this case we only setup :root as
# a config source so we just want to load that one. 
v = Vagrant::Vagrantfile.new(config_loader, [:root])

You can see another example of loading Vagrantfiles in the Vagrant::Environment module. There is the part that sets up the config loader

# ln 486 - 487
@config_loader.set(:home, home_vagrantfile) if home_vagrantfile
@config_loader.set(:root, root_vagrantfile) if root_vagrantfile

As you can see in these lines, 2 configuration sources are set for the config loader (conditionally).

And there is a part that uses that config loader to setup the Vagrantfile.

# ln 793
@vagrantfile ||= Vagrantfile.new(config_loader, [:home, :root])

Here a Vagrantfile is loaded using the sources :home and :root (that were setup for the config loader).

1 Like

Hello,

Firstly thanks a lot for your detailed answer. That really helped. Right now, we are able to load vagrantfile.

However, we have another problem right now. We can install plugin and add it in Vagrantfile.
Plugin gets triggered before machine up as expected… We authenticate to artifactory. However vagrant does not start downloading boxes after we authenticate.
If I add authentication block on top of Vagrantfile we can download boxes.

I believe I need to use an after hook to initialize downloading boxes.
I tried out couple different hooks but none of them worked so far.

Do you have any suggestions ?

Hello,

I actually found out this class for downloading boxes…

https://www.rubydoc.info/github/mitchellh/vagrant/Vagrant/Util/Downloader

This works if we statically pass url where the box is located in artifactory.
Now we need to pass “underlying box” for each machine to this downloader.

I tried to get by using “box” on machine_config method. However this returns nil if box is not in the disk.

https://www.rubydoc.info/github/mitchellh/vagrant/Vagrant/Vagrantfile#machine_config-instance_method

      # Setup the config loader
      config_loader = Vagrant::Config::Loader.new(
        Vagrant::Config::VERSIONS, Vagrant::Config::VERSIONS_ORDER)

      # Add the path to my_vagrantfile to the config loader
      config_loader.set(:root, vagrantfile_path)

      puts config_loader.load(["1"])

      # Create the Vagrantfile object. In this case we only setup :root as
      # a config source so we just want to load that one.
      v = Vagrant::Vagrantfile.new(config_loader, [:root])

      # We can't access config like this
      #v.inspect

      for machine_name in v.machine_names
        machine_config=v.machine_config(machine_name, env[:env].default_provider, env[:box_collection])

        machine_config[:box].name

This returns box names only if the boxes are already in the disk.

I wanted to make use of config on Vagrantfile class as well. However I couldn’t make use of it.
So what I want to know is there a way to get “underlying box” of a machine by using Vagrantfile class or some other class?
Thanks a lot.

Edit::

Ok I loaded config on Vagrantfile. I cant get underlying box there.

Hello,

Is there anyway to get ‘underlying box’ from Vagrantfile class instance. If we don’t have the box already installed in disk.
This “box” field on machine_config instance method returns nil if box is not installed in system.

Hi @soapy1, could you give us here some orientation? We are trying to get a list of boxes missing on disk to pass to the Downloader class.

Hey there,

sorry I’ve been neglecting the forum some :grimacing:

You can get a list of boxes that are on the machine using the Environment#boxes method. For example, you can reference the box list command implementation.

I’m not sure what information you have access to. But if you have a list of boxes that are required from the Vagrantfile then it should be pretty straightforward to compare the list of boxes available on disk (from Environment#boxes) to the list of required boxes to get a list of boxes that are missing.

Hey @soapy1, the problem is not getting the boxes on disk, but getting the required boxes from Vagrantfile.

Hmm, right. It looks like you can look at the machine method, instead of the the machine_config.

Then, it looks like from the machine you can use machine.config.vm.box to get the name of the box for the machine. That is how the handle_box action works.