Bug #21568
closedRequiring core libraries when already requiring multiple user defined libraries with the same name can error
Description
The simplest way to understand this error is that after requiring 2 or more files with the same name as some core libraries, any subsequent calls to require the core library will fail, ie
require 'foo/fiber'
require 'bar/fiber'
require 'fiber'
raises
cannot load such file -- fiber
More detailed instructions to repro this can be found in the code snippet below.
Noting that I am on a Mac, OS 15.6. This is reproducible across my team on a few different OS versions, all Mac. [EDIT]: I was also able to reproduce this on Debian Linux 12. The same underlying issue can also be seen in some other issues like: https://github.com/newrelic/newrelic-ruby-agent/issues/2001
The context this error is discovered in is that some gems may require their own libraries with the same name as core libraries, often wrapping the implementations of those core libraries to extend functionality, like the New Relic ruby agent does with fiber, for example (https://github.com/newrelic/newrelic-ruby-agent/tree/dev/lib/new_relic/agent/instrumentation/fiber).
Then, if 2 or more gems do something like that, and any other gem has require 'fiber'
in it to support old ruby versions where this was necessary, the error occurs.
Full Steps to Reproduce:
# Setup
# =====
# 1. Set this base_dir variable to be an absolute path to a dir
# or something that can be found in your $LOAD_PATH.
base_dir = # ex: "/Users/<you>/ruby_require_test"
# 2. Within that base directory create 2 directories named `foo` and `bar`
# 3. Within both `foo` and `bar` touch new files named
# - complex.rb
# - enumerator.rb
# - fiber.rb
# - rational.rb
# - ruby2_keywords.rb
# - set.rb
# - thread.rb
# Note - this list was generated by running:
# `ruby -ve 'p $LOADED_FEATURES.reject { |feature| %r|/| =~ feature }'`
# which returns: `["enumerator.so", "thread.rb", "fiber.so", "rational.so", "complex.so", "ruby2_keywords.rb"]`
# Reproducing the error:
# After requiring 2 (or more) user defined files with the same name as part of
# the ruby core library that is required as a bare filename, requiring that part of
# the core library will error, ex: `cannot load such file -- fiber`.
# The erroring require statements are all .so files
begin
require "#{base_dir}/foo/complex"
require "#{base_dir}/bar/complex"
require 'complex'
rescue LoadError => e
puts e.message
end
begin
require "#{base_dir}/foo/enumerator"
require "#{base_dir}/bar/enumerator"
require 'enumerator'
rescue LoadError => e
puts e.message
end
begin
require "#{base_dir}/foo/fiber"
require "#{base_dir}/bar/fiber"
require 'fiber'
rescue LoadError => e
puts e.message
end
begin
require "#{base_dir}/foo/rational"
require "#{base_dir}/bar/rational"
require 'rational'
rescue LoadError => e
puts e.message
end
# The following examples do not error.
# The difference seems to be that the core lib files are defined as .rb not .so
begin
require "#{base_dir}/foo/ruby2_keywords"
require "#{base_dir}/bar/ruby2_keywords"
require 'ruby2_keywords'
rescue LoadError => e
puts e.message
end
begin
require "#{base_dir}/foo/set"
require "#{base_dir}/bar/set"
require 'set'
rescue LoadError => e
puts e.message
end
begin
require "#{base_dir}/foo/thread"
require "#{base_dir}/bar/thread"
require 'thread'
rescue LoadError => e
puts e.message
end
Updated by alexalexgriffith (Alex Griffith) 22 days ago
ยท Edited
- Subject changed from Requiring core libraries when already requiring mutliple user defined libraries with the same name can error to Requiring core libraries when already requiring multiple user defined libraries with the same name can error
Updated by alexalexgriffith (Alex Griffith) 1 day ago
- Description updated (diff)
- ruby -v changed from ruby 3.1.7 to ruby 3.4.6
Updated to indicate this is reproducible on the latest ruby version as well as on Debian Linux.
Updated by byroot (Jean Boussier) about 16 hours ago
Hi, could you provide a fully executable reproduction? One possibility is just to create a test repository we can just clone and run.
Otherwise I'm worried I may not fully understand how the setup should be.
Updated by mame (Yusuke Endoh) about 14 hours ago
Here is a small repro:
$LOADED_FEATURES.replace(["foo.so", "a/foo.rb"])
require "foo" #=> false
$LOADED_FEATURES.replace(["foo.so", "a/foo.rb", "b/foo.rb"])
require "foo" #=> cannot load such file -- foo
Indeed, looks like something is wrong. Since it reproduces in miniruby, I don't think rubygems is involved.
Updated by nobu (Nobuyoshi Nakada) about 14 hours ago
- Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN to 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
Updated by mame (Yusuke Endoh) about 13 hours ago
I created a patch. https://github.com/ruby/ruby/pull/14702
Updated by mame (Yusuke Endoh) about 13 hours ago
- Status changed from Open to Closed
Applied in changeset git|7ae67e8f6ad6e7fd0677b28a7a10961f79d55495.
load.c: Fix dest and src of MEMMOVE
When multiple files with the same name are required, the features_index
hash stores the indexes in $LOADED_FEATURES
array into a darray.
The dest and src arguments for MEMMOVE
were wrongly reversed when
inserting a new index in the darray.
[Bug #21568]
Updated by byroot (Jean Boussier) about 13 hours ago
Nice catch, looks like it was introduced in 3.1.