Project

General

Profile

Actions

Bug #21568

closed

Requiring core libraries when already requiring multiple user defined libraries with the same name can error

Added by alexalexgriffith (Alex Griffith) 22 days ago. Updated about 13 hours ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:123207]

Description

The simplest way to understand this error is that after requiring 2 or more files with the same name as some core libraries, any subsequent calls to require the core library will fail, ie

require 'foo/fiber'
require 'bar/fiber'
require 'fiber'

raises

cannot load such file -- fiber

More detailed instructions to repro this can be found in the code snippet below.

Noting that I am on a Mac, OS 15.6. This is reproducible across my team on a few different OS versions, all Mac. [EDIT]: I was also able to reproduce this on Debian Linux 12. The same underlying issue can also be seen in some other issues like: https://github.com/newrelic/newrelic-ruby-agent/issues/2001

The context this error is discovered in is that some gems may require their own libraries with the same name as core libraries, often wrapping the implementations of those core libraries to extend functionality, like the New Relic ruby agent does with fiber, for example (https://github.com/newrelic/newrelic-ruby-agent/tree/dev/lib/new_relic/agent/instrumentation/fiber).

Then, if 2 or more gems do something like that, and any other gem has require 'fiber' in it to support old ruby versions where this was necessary, the error occurs.

Full Steps to Reproduce:

# Setup
# =====
# 1. Set this base_dir variable to be an absolute path to a dir
#    or something that can be found in your $LOAD_PATH.
base_dir = # ex: "/Users/<you>/ruby_require_test"
# 2. Within that base directory create 2 directories named `foo` and `bar`
# 3. Within both `foo` and `bar` touch new files named
#    - complex.rb 
#    - enumerator.rb 
#    - fiber.rb 
#    - rational.rb 
#    - ruby2_keywords.rb 
#    - set.rb 
#    - thread.rb 
# Note - this list was generated by running:
#   `ruby -ve 'p $LOADED_FEATURES.reject { |feature| %r|/| =~ feature }'`
#   which returns: `["enumerator.so", "thread.rb", "fiber.so", "rational.so", "complex.so", "ruby2_keywords.rb"]`


# Reproducing the error:
# After requiring 2 (or more) user defined files with the same name as part of
# the ruby core library that is required as a bare filename, requiring that part of
# the core library will error, ex: `cannot load such file -- fiber`.
# The erroring require statements are all .so files

begin
  require "#{base_dir}/foo/complex"
  require "#{base_dir}/bar/complex"
  require 'complex'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/enumerator"
  require "#{base_dir}/bar/enumerator"
  require 'enumerator'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/fiber"
  require "#{base_dir}/bar/fiber"
  require 'fiber'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/rational"
  require "#{base_dir}/bar/rational"
  require 'rational'
rescue LoadError => e
  puts e.message
end


# The following examples do not error.
# The difference seems to be that the core lib files are defined as .rb not .so
begin
  require "#{base_dir}/foo/ruby2_keywords"
  require "#{base_dir}/bar/ruby2_keywords"
  require 'ruby2_keywords'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/set"
  require "#{base_dir}/bar/set"
  require 'set'
rescue LoadError => e
  puts e.message
end

begin
  require "#{base_dir}/foo/thread"
  require "#{base_dir}/bar/thread"
  require 'thread'
rescue LoadError => e
  puts e.message
end

Updated by alexalexgriffith (Alex Griffith) 22 days ago ยท Edited

  • Subject changed from Requiring core libraries when already requiring mutliple user defined libraries with the same name can error to Requiring core libraries when already requiring multiple user defined libraries with the same name can error

Updated by alexalexgriffith (Alex Griffith) 1 day ago

  • Description updated (diff)
  • ruby -v changed from ruby 3.1.7 to ruby 3.4.6

Updated to indicate this is reproducible on the latest ruby version as well as on Debian Linux.

Updated by byroot (Jean Boussier) about 16 hours ago

Hi, could you provide a fully executable reproduction? One possibility is just to create a test repository we can just clone and run.

Otherwise I'm worried I may not fully understand how the setup should be.

Updated by mame (Yusuke Endoh) about 14 hours ago

Here is a small repro:

$LOADED_FEATURES.replace(["foo.so", "a/foo.rb"])
require "foo" #=> false

$LOADED_FEATURES.replace(["foo.so", "a/foo.rb", "b/foo.rb"])
require "foo" #=> cannot load such file -- foo

Indeed, looks like something is wrong. Since it reproduces in miniruby, I don't think rubygems is involved.

Actions #5

Updated by nobu (Nobuyoshi Nakada) about 14 hours ago

  • Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN to 3.2: REQUIRED, 3.3: REQUIRED, 3.4: REQUIRED
Actions #7

Updated by mame (Yusuke Endoh) about 13 hours ago

  • Status changed from Open to Closed

Applied in changeset git|7ae67e8f6ad6e7fd0677b28a7a10961f79d55495.


load.c: Fix dest and src of MEMMOVE

When multiple files with the same name are required, the features_index
hash stores the indexes in $LOADED_FEATURES array into a darray.
The dest and src arguments for MEMMOVE were wrongly reversed when
inserting a new index in the darray.

[Bug #21568]

Updated by byroot (Jean Boussier) about 13 hours ago

Nice catch, looks like it was introduced in 3.1.

Actions

Also available in: Atom PDF

Like1
Like0Like0Like0Like1Like0Like1Like0Like0