Bug #21568: Requiring core libraries when already requiring multiple user defined libraries with the same name can error

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.4.6
• Backport: 3.2: REQUIRED, 3.3: DONE, 3.4: DONE

[alexalexgriffith (Alex Griffith)]

The simplest way to understand this error is that after requiring 2 or more files with the same name as some core libraries, any subsequent calls to require the core library will fail, ie

require 'foo/fiber'
require 'bar/fiber'
require 'fiber'

raises

cannot load such file -- fiber

More detailed instructions to repro this can be found in the code snippet below.

Noting that I am on a Mac, OS 15.6. This is reproducible across my team on a few different OS versions, all Mac. [EDIT]: I was also able to reproduce this on Debian Linux 12. The same underlying issue can also be seen in some other issues like: https://github.com/newrelic/newrelic-ruby-agent/issues/2001

The context this error is discovered in is that some gems may require their own libraries with the same name as core libraries, often wrapping the implementations of those core libraries to extend functionality, like the New Relic ruby agent does with fiber, for example (https://github.com/newrelic/newrelic-ruby-agent/tree/dev/lib/new_relic/agent/instrumentation/fiber).

Then, if 2 or more gems do something like that, and any other gem has require 'fiber' in it to support old ruby versions where this was necessary, the error occurs.

Full Steps to Reproduce:

# Setup
# =====
# 1. Set this base_dir variable to be an absolute path to a dir
#    or something that can be found in your $LOAD_PATH.
base_dir = # ex: "/Users/<you>/ruby_require_test"
# 2. Within that base directory create 2 directories named `foo` and `bar`
# 3. Within both `foo` and `bar` touch new files named
#    - complex.rb 
#    - enumerator.rb 
#    - fiber.rb 
#    - rational.rb 
#    - ruby2_keywords.rb 
#    - set.rb 
#    - thread.rb 
# Note - this list was generated by running:
#   `ruby -ve 'p $LOADED_FEATURES.reject { |feature| %r|/| =~ feature }'`
#   which returns: `["enumerator.so", "thread.rb", "fiber.so", "rational.so", "complex.so", "ruby2_keywords.rb"]`


# Reproducing the error:
# After requiring 2 (or more) user defined files with the same name as part of
# the ruby core library that is required as a bare filename, requiring that part of
# the core library will error, ex: `cannot load such file -- fiber`.
# The erroring require statements are all .so files

begin
  require "#{base_dir}/foo/complex"
  require "#{base_dir}/bar/complex"
  require 'complex'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/enumerator"
  require "#{base_dir}/bar/enumerator"
  require 'enumerator'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/fiber"
  require "#{base_dir}/bar/fiber"
  require 'fiber'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/rational"
  require "#{base_dir}/bar/rational"
  require 'rational'
rescue LoadError => e
  puts e.message
end


# The following examples do not error.
# The difference seems to be that the core lib files are defined as .rb not .so
begin
  require "#{base_dir}/foo/ruby2_keywords"
  require "#{base_dir}/bar/ruby2_keywords"
  require 'ruby2_keywords'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/set"
  require "#{base_dir}/bar/set"
  require 'set'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/thread"
  require "#{base_dir}/bar/thread"
  require 'thread'
rescue LoadError => e
  puts e.message
end

[alexalexgriffith (Alex Griffith)]

• Subject changed from Requiring core libraries when already requiring mutliple user defined libraries with the same name can error to Requiring core libraries when already requiring multiple user defined libraries with the same name can error

[alexalexgriffith (Alex Griffith)]

• Description updated (diff)
• ruby -v changed from ruby 3.1.7 to ruby 3.4.6

Updated to indicate this is reproducible on the latest ruby version as well as on Debian Linux.

[byroot (Jean Boussier)]

Hi, could you provide a fully executable reproduction? One possibility is just to create a test repository we can just clone and run.

Otherwise I'm worried I may not fully understand how the setup should be.

[mame (Yusuke Endoh)]

Here is a small repro:

$LOADED_FEATURES.replace(["foo.so", "a/foo.rb"])
require "foo" #=> false

$LOADED_FEATURES.replace(["foo.so", "a/foo.rb", "b/foo.rb"])
require "foo" #=> cannot load such file -- foo

Indeed, looks like something is wrong. Since it reproduces in miniruby, I don't think rubygems is involved.

[nobu (Nobuyoshi Nakada)]

• Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN to 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED

[mame (Yusuke Endoh)]

I created a patch. https://github.com/ruby/ruby/pull/14702

[mame (Yusuke Endoh)]

• Status changed from Open to Closed

Applied in changeset git|7ae67e8f6ad6e7fd0677b28a7a10961f79d55495.

---

load.c: Fix dest and src of MEMMOVE

When multiple files with the same name are required, the features_index
hash stores the indexes in $LOADED_FEATURES array into a darray.
The dest and src arguments for MEMMOVE were wrongly reversed when
inserting a new index in the darray.

[Bug #21568]

[byroot (Jean Boussier)]

Nice catch, looks like it was introduced in 3.1.

[k0kubun (Takashi Kokubun)]

• Backport changed from 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED to 3.2: REQUIRED, 3.3: REQUIRED, 3.4: DONE

ruby_3_4 ef3ce711fb29f7273216edb2fe41bb2604c835c3 merged revision(s) 7ae67e8f6ad6e7fd0677b28a7a10961f79d55495.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.2: REQUIRED, 3.3: REQUIRED, 3.4: DONE to 3.2: REQUIRED, 3.3: DONE, 3.4: DONE

ruby_3_3 27a1df7506ba566d08a536e93910fff574cd6d56 merged revision(s) 7ae67e8f6ad6e7fd0677b28a7a10961f79d55495.