Vagrant up error on Ubuntu 22.04: pkeys are immutable on OpenSSL 3.0

giacomo · April 26, 2022, 9:44pm

Hi,

I recently ugraded from Ubuntu 20.04 to 22.04 (specifically pop os). I noticed vagrant doesn’t work anymore.

It seems to be an issue with the rubygem net-ssh giving error “pkeys are immutable on OpenSSL 3.0”.

Here’s the complete output of vagrant up:

`Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'ubuntu/focal64' version '20220324.0.0' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
/usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/kex/ecdh_sha2_nistp256.rb:21:in `generate_key!': pkeys are immutable on OpenSSL 3.0 (OpenSSL::PKey::PKeyError)
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/kex/ecdh_sha2_nistp256.rb:21:in `generate_key'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/kex/abstract.rb:32:in `initialize'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/algorithms.rb:437:in `new'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/algorithms.rb:437:in `exchange_keys'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/algorithms.rb:245:in `proceed!'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/algorithms.rb:184:in `accept_kexinit'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:210:in `block in poll_message'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:190:in `loop'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:190:in `poll_message'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:225:in `block in wait'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:223:in `loop'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:223:in `wait'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh/transport/session.rb:90:in `initialize'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh.rb:251:in `new'
	from /usr/share/rubygems-integration/all/gems/net-ssh-6.1.0/lib/net/ssh.rb:251:in `start'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/communicators/ssh/communicator.rb:467:in `block (2 levels) in connect'
	from /usr/lib/ruby/3.0.0/timeout.rb:97:in `block in timeout'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `block in catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:112:in `timeout'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/communicators/ssh/communicator.rb:433:in `block in connect'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/util/retryable.rb:17:in `retryable'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/communicators/ssh/communicator.rb:432:in `connect'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/communicators/ssh/communicator.rb:87:in `block in wait_for_ready'
	from /usr/lib/ruby/3.0.0/timeout.rb:97:in `block in timeout'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `block in catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:35:in `catch'
	from /usr/lib/ruby/3.0.0/timeout.rb:112:in `timeout'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/plugins/communicators/ssh/communicator.rb:63:in `wait_for_ready'
	from /usr/share/rubygems-integration/all/gems/vagrant-2.2.19/lib/vagrant/action/builtin/wait_for_communicator.rb:16:in `block in call`

The only relevant thing I can find online is this.

As far as I understand (from this stackoverflow question) the problem is OpenSSL 3.0 breaking ruby-net-ssh. Is there any workaround possible here? Or should I just wait for a fix from Ubuntu/ruby-net-ssh?

Thanks in advance for the help

rcitton · April 27, 2022, 10:03pm

as reported on https://github.com/hashicorp/vagrant/issues/12751
A workaround is to uninstall vagrant with
‘sudo apt-get remove vagrant’
and to install a .deb file found at the bottom of this page :
https://www.vagrantup.com/downloads

giacomo · April 29, 2022, 9:14am

Thanks that fixed it.

rubo77 · April 29, 2022, 10:59am

the workaround didn’t work for me completely, the error messages are still there, but the existing virtual machine starts OK anyway!
A newly provisioned machine does not start completely though: I can vagrant ssh into it, but the provisioning does not work.

(also after upgrade from 21.10 to 22.04)

chrispito · May 5, 2022, 11:16am

The workaround also worked for me.

Ubuntu: 22.04
Vagrant 2.2.19
VirtualBox 6.1.32_Ubuntu r149290

Thanks

elreydetoda · May 6, 2022, 10:49pm

tl;dr: do sudo apt install vagrant=2.2.19 till hashicorp revs their next version of Vagrant, then the repos will put that as the highest version’ed one.

So, I had this same issue as well, but I had the hashicorp repos installed to install vagrant. I knew that the .deb file should be the same, and I figured out what is wrong (at least for me on Pop_OS! 22.04).

It looks like Canonical’s repos are acting like they have a higher version than HashiCorp’s. Whenever I did an apt info vagrant it showed me Canonical’s version not the HashiCorp one.:

$  apt info vagrant   
Package: vagrant
Version: 2.2.19+dfsg-1ubuntu1
Priority: optional
Section: universe/admin
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Ruby Team <pkg-ruby-extras-maintainers@lists.alioth.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 3,559 kB
Depends: libarchive-tools, curl, openssh-client, rsync, ruby, ruby-bcrypt-pbkdf, ruby-childprocess, ruby-ed25519, ruby-erubi, ruby-i18n, ruby-listen, ruby-log4r, ruby-mime-types, ruby-net-ssh, ruby-net-sftp, ruby-net-scp, ruby-rexml, ruby-zip, ruby-vagrant-cloud
Recommends: vagrant-libvirt
Suggests: virtualbox (>= 4.0)
Breaks: virtualbox (>= 6.2)
Homepage: https://www.vagrantup.com
Ruby-Versions: all
Download-Size: 473 kB
APT-Sources: http://us.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
Description: Tool for building and distributing virtualized development environments
 This package provides the tools to create and configure lightweight,
 reproducible, and portable virtual environments.
 .
 Vagrant upstream uses Oracle’s VirtualBox by default to create its virtual
 machines. On Debian, Vagrant will use libvirt/KVM by default as VirtualBox is
 not part of Debian main, but will use VirtualBox if it's installed.

N: There are 11 additional records. Please use the '-a' switch to see them.

Whenever I did the -a like it recommended. I saw all the HasihiCorp versions:

$  apt info vagrant -a                                
Package: vagrant
Version: 2.2.19+dfsg-1ubuntu1
Priority: optional
Section: universe/admin
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Ruby Team <pkg-ruby-extras-maintainers@lists.alioth.debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 3,559 kB
Depends: libarchive-tools, curl, openssh-client, rsync, ruby, ruby-bcrypt-pbkdf, ruby-childprocess, ruby-ed25519, ruby-erubi, ruby-i18n, ruby-listen, ruby-log4r, ruby-mime-types, ruby-net-ssh, ruby-net-sftp, ruby-net-scp, ruby-rexml, ruby-zip, ruby-vagrant-cloud
Recommends: vagrant-libvirt
Suggests: virtualbox (>= 4.0)
Breaks: virtualbox (>= 6.2)
Homepage: https://www.vagrantup.com
Ruby-Versions: all
Download-Size: 473 kB
APT-Sources: http://us.archive.ubuntu.com/ubuntu jammy/universe amd64 Packages
Description: Tool for building and distributing virtualized development environments
 This package provides the tools to create and configure lightweight,
 reproducible, and portable virtual environments.
 .
 Vagrant upstream uses Oracle’s VirtualBox by default to create its virtual
 machines. On Debian, Vagrant will use libvirt/KVM by default as VirtualBox is
 not part of Debian main, but will use VirtualBox if it's installed.

Package: vagrant
Version: 2.2.19
Priority: extra
Section: default
Maintainer: HashiCorp <support@hashicorp.com>
Installed-Size: 117 MB
Homepage: https://www.vagrantup.com/
License: MIT
Vendor: HashiCorp
Download-Size: 41.5 MB
APT-Manual-Installed: yes
APT-Sources: https://apt.releases.hashicorp.com jammy/main amd64 Packages
Description: Vagrant is a tool for building and distributing development environments.

Package: vagrant
Version: 2.2.18
Priority: extra
...

So, you can force apt to install a specific version of a package and I forced apt to install HashiCorp’s version: sudo apt install vagrant=2.2.19

I personally use libvirt before Virtualbox, so after I did the forced apt install I did sudo apt install vagrant-libvirt and everything seems to be working for me now.

rubo77 · May 19, 2022, 6:13pm

seems like the bug was fixed in the package ruby-net-ssh - 1:6.1.0-2ubuntu0.1

Topic		Replies	Views
Vagrant builtin ssh (at `vagrant up`) still problems with key exchange Vagrant	0	532	March 8, 2022
Error in "vagrant up" on Mac Vagrant	4	1803	January 6, 2021
Vagrant Private SSH Key Vagrant	0	220	April 8, 2023
The SSH Time out issue - with VirtualBox I think Vagrant	0	275	November 16, 2023
Problem with vagrant ssh Vagrant	1	340	August 27, 2023

Vagrant up error on Ubuntu 22.04: pkeys are immutable on OpenSSL 3.0

Related topics