Misc #22275: Ruby::Box support plan for RubyGems and Bundler

• Status: Open
• Assignee: -

[hsbt (Hiroshi SHIBATA)]

This is a tracking issue for making RubyGems and Bundler work with Ruby::Box, so that reviewers can see the whole picture and what to look at next in one place.

Goal

The goal is to realize Feature #13847 with Ruby::Box: loading a specific version of a gem, including default gems and C extensions, isolated from the copy that RubyGems/Bundler themselves use. Vendoring with namespace rewriting has been our workaround for pure-Ruby gems for years; Box removes that limitation.

As a further step, we want RubyGems/Bundler to activate different gem versions per box, which enables the use cases already proposed around Box: gradual dependency upgrades, plugin systems with conflicting dependencies, and multiple applications in one process.

First milestone

Before designing such APIs, Box needs to leave experimental status. Our first milestone is that Rails and RubyGems/Bundler themselves work under RUBY_BOX=1, and that the ruby/rubygems repository runs its test suites with Box enabled continuously so we do not regress.

Current status

A rails new application now boots, serves scaffold CRUD requests, and reloads code in development mode once ruby/ruby#18534 is applied. The rubygems test suite runs with 0 failures under RUBY_BOX=1 (ruby/rubygems#9826, open), with pending marks referencing the unfixed core bugs below.

Fixed

ruby/ruby

• ruby/ruby#18218 / ruby/ruby#18219 — BUNDLER_SETUP was consumed outside the main box (Bug #22123). Shipped in 4.0.7.
• ruby/ruby#18509 — box-local extension DLLs on Windows were unloaded too early; now deferred.

ruby/rubygems

• ruby/rubygems#9809 — Bundler evaluated gemspecs through TOPLEVEL_BINDING, which always belongs to the main box. It now uses a binding in the box Bundler is loaded in, so gemspecs resolve the right Gem::Specification.
• ruby/rubygems#9810 — the gem CLI died under RUBY_BOX=1. Marshal-based deep copies could not resolve Gem:: constants across boxes and were replaced with a plain deep dup, and RUBY_BOX is now stripped from extension build subprocesses to avoid an RbConfig.expand infinite recursion. Includes a CLI canary test that runs under RUBY_BOX=1.

Open pull requests

All review-requested to @tagomoris (Satoshi Tagomori):

• ruby/ruby#18534 — autoload-triggered require bypasses the box's Kernel#require (Bug #21830). This alone takes Rails from 500 on every request to fully working.
• ruby/ruby#18546 — Symbol#to_proc ignores box-local method definitions (Bug #22015, revives ruby/ruby#16865). This alone unblocks Bundler's spec harness, which dies in rspec-core's &:shellsplit before running a single spec.
• ruby/ruby#18544 — Marshal.load resolves classes in the root box (Bug #22090); an in-process round-trip of Gem::Version fails today.
• ruby/ruby#18536 — the box extension copy embeds the full path in the temporary filename, exceeding NAME_MAX on deep paths, and the name is predictable (Bug #22110).
• ruby/ruby#18535 — with --disable=gems, modules prepended to Kernel in a user box end up behind Kernel in the ancestry, so the prepended method is never dispatched.

Remaining problem without a fix

Bug #21867 turned out to be broader than warn: reassigning $stdout/$stderr is invisible to builtin writers under Box, which breaks output-capturing test helpers everywhere. We could not find a harness-side workaround.

Critical path

The most critical items for the milestone are ruby/ruby#18534 (everything Zeitwerk-based, i.e. Rails, depends on it) and ruby/ruby#18546 (Bundler's entire spec suite is gated behind it). After those, ruby/ruby#18544 and Bug #21867 let us drop most of the pending marks in the rubygems suite.

[hsbt (Hiroshi SHIBATA)]

• Description updated (diff)

[hsbt (Hiroshi SHIBATA)]

• Related to Feature #13847: Gem activated problem for default gems added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #22123: Ruby::Box + `BUNDLER_SETUP` can evaluate gemspecs before main-box RubyGems initialization added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #21830: Ruby::Box and Kernel#require added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #22015: Success without RUBY_BOX=1, Failure with RUBY_BOX=1 added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #22090: Enabling Ruby::Box breaks Marshal.load added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #22110: Ruby::Box C-extension loading can be denied by predictable temporary filename added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #21867: enabling Ruby::Box changes puts/warn to no longer use $stdout/$stderr added