Bug #15500: Behavior of require method in 2.5 is different from 2.4 and 2.6

• Status: Closed
• Assignee: hsbt (Hiroshi SHIBATA)
• Target version: -
• ruby -v: ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
• Backport: 2.4: DONTNEED, 2.5: DONE, 2.6: DONE

[mrkn (Kenta Murata)]

On Ruby 2.5, require 'bigdecimal' does not load bigdecimal.so in gem-installed version of bigdecimal without specifying the version by gem method.
You can examine this phenomenon by the following code.

require 'bigdecimal'
p $LOADED_FEATURES.grep(/bigdecimal/).first

After installing bigdecimal-1.4.2 by gem install bigdecimal command, this code (saved as t.rb) show the following results:

On Ruby 2.4.5

$ ruby-2.4.5 -v t.rb
ruby 2.4.5p335 (2018-10-18 revision 65137) [x86_64-darwin18]
"/Users/mrkn/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/bigdecimal-1.4.2/lib/bigdecimal.bundle"

On Ruby 2.5.3:

$ ruby-2.5.3 -v t.rb
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
"/Users/mrkn/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin18/bigdecimal.bundle"

On Ruby 2.6.0:

$ ruby-2.6.0 -v t.rb
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-darwin18]
"/Users/mrkn/.rbenv/versions/2.6.0/lib/ruby/gems/2.6.0/gems/bigdecimal-1.4.2/lib/bigdecimal.bundle"

On Ruby 2.5.3, with gem 'bigdecimal', '1.4.2', the correct file can be loaded.

$ ( echo "gem 'bigdecimal', '1.4.2'"; cat t.rb ) | ruby-2.5.3 -v -
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
"/Users/mrkn/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal.bundle"

[mrkn (Kenta Murata)]

• Description updated (diff)

[marcandre (Marc-Andre Lafortune)]

• Related to Misc #15486: Default gems README.md added

[marcandre (Marc-Andre Lafortune)]

I was expecting the behavior in 2.5. That's what is correct, right?

I tested with matrix gem, and 2.4 behaves like 2.5, which is to say it loads the gem only if gem 'matrix' is called. Note that 2.4 didn't ship with default matrix gem (it was std lib), but it did ship with defaut bigdecimal gem.

I also realized that it's not possible to load the 0.1.0 matrix gem in 2.6.0 because it shipped with that version of the local gem. That in itself is a problem, as the public gem matrix v0.1.0 is actually older than the local gem that has shipped with Ruby 2.6.0. I will release v1.0.0 to avoid this issue, and I will have to be careful when mirroring the gem.

[mrkn (Kenta Murata)]

I think the behavior of 2.5 can be thought of a bug because the behavior for .so file isn't consistent with one for .rb file.
You can see the inconsistency in the results below.

$ ruby -rbigdecimal/util -ve 'p $LOADED_FEATURES.grep(/bigdecimal/)'
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
["/usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal.so", "/usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal.rb", "/usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal/util.so", "/usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal/util.rb"]

In this case, bigdecimal/util is required, and the gem-installed version of bigdecimal/util.rb is loaded.
You can see the gem-installed version of bigdecimal.so is also loaded.

The behavior is changed when bigdecimal is required before bigdecimal/util.

$ ruby -rbigdecimal -rbigdecimal/util -ve 'p $LOADED_FEATURES.grep(/bigdecimal/)'
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
/usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal.rb:7: warning: method redefined; discarding old new
Traceback (most recent call last):
	5: from /usr/local/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	4: from /usr/local/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	3: from /usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal/util.rb:9:in `<top (required)>'
	2: from /usr/local/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:34:in `require'
	1: from /usr/local/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:130:in `rescue in require'
/usr/local/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:130:in `require': /usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal/util.so: undefined symbol: rmpd_util_str_to_d - /usr/local/lib/ruby/gems/2.5.0/gems/bigdecimal-1.4.2/lib/bigdecimal/util.so (LoadError)

In this case, LoadError is occurred by the symbol resolution failure of bigdecimal/util.so. This file is newly introduced since bigdecimal-1.4.0.
Although bigdecimal.so in bigdecimal-1.4.2 has rmpd_util_str_to_d, it isn't loaded because the default-gem version of bigdecimal.so is loaded by -rbigdecimal option.

The difference between the default-gem and gem-installed versions is whether bigdecimal.rb exists. Although I've not investigated the implementation of Ruby 2.5's require` method yet, I consider that Ruby 2.5 won't search in the gem-installed version if there is .so file but there isn't .rb file.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned
• Assignee set to hsbt (Hiroshi SHIBATA)

[hsbt (Hiroshi SHIBATA)]

I could reproduce them. But I also reproduce it with Ruby 2.5.3 with RubyGems 2.6.14.3 bundled with Ruby 2.4.

~ > ruby -v
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin18]
~ > gem -v
2.6.14.3
~ > gem list bigdecimal

*** LOCAL GEMS ***

bigdecimal (1.4.3, default: 1.3.4)
~ > ruby -e "require 'bigdecimal'; p \$LOADED_FEATURES.grep(/bigdecimal/).first"
"/Users/hsbt/.rbenv/versions/2.5.3/lib/ruby/2.5.0/x86_64-darwin18/bigdecimal.bundle"

It seems this issue produced by Ruby core without RubyGems maybe.

[mrkn (Kenta Murata)]

• Related to Bug #15545: Backport r58403 added

[hsbt (Hiroshi SHIBATA)]

• Status changed from Assigned to Closed

Applied in changeset trunk|r66867.

---

Revert r58345 and r58371.

These changes break the behavior of default gems. Bug #13428 says
r58345 is reasonable because gemspec file is installed by to_ruby_for_cache
method. But I revert to_ruby_for_cache in rbinstall.rb at r58403.
There is no reason that we apply r58345 now.

But I'm not sure about gemspec of default gems affects standalone gems.
I'm going to investigate it on rubygems/rubygems.

[Bug #15500][ruby-core:90867]

[hsbt (Hiroshi SHIBATA)]

@mrkn (Kenta Murata)

Can you try with Ruby 2.5 applied r66867?

[hsbt (Hiroshi SHIBATA)]

• Backport changed from 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN to 2.4: DONTNEED, 2.5: REQUIRED, 2.6: REQUIRED

[naruse (Yui NARUSE)]

• Backport changed from 2.4: DONTNEED, 2.5: REQUIRED, 2.6: REQUIRED to 2.4: DONTNEED, 2.5: REQUIRED, 2.6: DONE

ruby_2_6 r66913 merged revision(s) 66867.

[mrkn (Kenta Murata)]

@hsbt (Hiroshi SHIBATA)
I've confirmed that Ruby 2.5 with r66867 loads bigdecimal installed by gem, correctly.

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #15469: Ruby2.6 included `bundler` does not handle specified `csv` gem. added

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.4: DONTNEED, 2.5: REQUIRED, 2.6: DONE to 2.4: DONTNEED, 2.5: DONE, 2.6: DONE

ruby_2_5 r67013 merged revision(s) 66867.