Misc #22275
Updated by hsbt (Hiroshi SHIBATA) about 1 month ago
This is a tracking issue for making RubyGems and Bundler work with Ruby::Box, so that reviewers can see the whole picture and what to look at next in one place. ## Goal The goal is to realize [Feature #13847](https://bugs.ruby-lang.org/issues/13847) https://bugs.ruby-lang.org/issues/13847 with Ruby::Box: loading a specific version of a gem, including default gems and C extensions, isolated from the copy that RubyGems/Bundler themselves use. Vendoring with namespace rewriting has been our workaround for pure-Ruby gems for years; Box removes that limitation. As a further step, we want RubyGems/Bundler to activate different gem versions per box, which enables the use cases already proposed around Box: gradual dependency upgrades, plugin systems with conflicting dependencies, and multiple applications in one process. ## First milestone Before designing such APIs, Box needs to leave experimental status. Our first milestone is that Rails and RubyGems/Bundler themselves work under `RUBY_BOX=1`, and that the ruby/rubygems repository runs its test suites with Box enabled continuously so we do not regress. ## Current status A `rails new` application now boots, serves scaffold CRUD requests, and reloads code in development mode once [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) https://github.com/ruby/ruby/pull/18534 is applied. The rubygems test suite runs with 0 failures under `RUBY_BOX=1` ([ruby/rubygems#9826](https://github.com/ruby/rubygems/pull/9826), (https://github.com/ruby/rubygems/pull/9826, open), with pending marks referencing the unfixed core bugs below. ## Fixed ### ruby/ruby * [ruby/ruby#18218](https://github.com/ruby/ruby/pull/18218) https://github.com/ruby/ruby/pull/18218 / [ruby/ruby#18219](https://github.com/ruby/ruby/pull/18219) https://github.com/ruby/ruby/pull/18219 — `BUNDLER_SETUP` was consumed outside the main box ([Bug #22123](https://bugs.ruby-lang.org/issues/22123)). (https://bugs.ruby-lang.org/issues/22123). Shipped in 4.0.7. * [ruby/ruby#18509](https://github.com/ruby/ruby/pull/18509) https://github.com/ruby/ruby/pull/18509 — box-local extension DLLs on Windows were unloaded too early; now deferred. ### ruby/rubygems * [ruby/rubygems#9809](https://github.com/ruby/rubygems/pull/9809) https://github.com/ruby/rubygems/pull/9809 — Bundler evaluated gemspecs through `TOPLEVEL_BINDING`, which always belongs to the main box. It now uses a binding in the box Bundler is loaded in, so gemspecs resolve the right `Gem::Specification`. * [ruby/rubygems#9810](https://github.com/ruby/rubygems/pull/9810) https://github.com/ruby/rubygems/pull/9810 — the `gem` CLI died under `RUBY_BOX=1`. `Marshal`-based deep copies could not resolve `Gem::` constants across boxes and were replaced with a plain deep dup, and `RUBY_BOX` is now stripped from extension build subprocesses to avoid an `RbConfig.expand` infinite recursion. Includes a CLI canary test that runs under `RUBY_BOX=1`. ## Open pull requests All review-requested to @tagomoris: * [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) https://github.com/ruby/ruby/pull/18534 — autoload-triggered require bypasses the box's `Kernel#require` ([Bug #21830](https://bugs.ruby-lang.org/issues/21830)). (https://bugs.ruby-lang.org/issues/21830). This alone takes Rails from 500 on every request to fully working. * [ruby/ruby#18546](https://github.com/ruby/ruby/pull/18546) https://github.com/ruby/ruby/pull/18546 — `Symbol#to_proc` ignores box-local method definitions ([Bug #22015](https://bugs.ruby-lang.org/issues/22015), (https://bugs.ruby-lang.org/issues/22015, revives [ruby/ruby#16865](https://github.com/ruby/ruby/pull/16865)). #16865). This alone unblocks Bundler's spec harness, which dies in rspec-core's `&:shellsplit` before running a single spec. * [ruby/ruby#18544](https://github.com/ruby/ruby/pull/18544) https://github.com/ruby/ruby/pull/18544 — `Marshal.load` resolves classes in the root box ([Bug #22090](https://bugs.ruby-lang.org/issues/22090)); (https://bugs.ruby-lang.org/issues/22090); an in-process round-trip of `Gem::Version` fails today. * [ruby/ruby#18536](https://github.com/ruby/ruby/pull/18536) https://github.com/ruby/ruby/pull/18536 — the box extension copy embeds the full path in the temporary filename, exceeding NAME_MAX on deep paths, and the name is predictable ([Bug #22110](https://bugs.ruby-lang.org/issues/22110)). (https://bugs.ruby-lang.org/issues/22110). * [ruby/ruby#18535](https://github.com/ruby/ruby/pull/18535) https://github.com/ruby/ruby/pull/18535 — with `--disable=gems`, modules prepended to `Kernel` in a user box end up behind `Kernel` in the ancestry, so the prepended method is never dispatched. ## Remaining problem without a fix [Bug #21867](https://bugs.ruby-lang.org/issues/21867) https://bugs.ruby-lang.org/issues/21867 turned out to be broader than `warn`: reassigning `$stdout`/`$stderr` is invisible to builtin writers under Box, which breaks output-capturing test helpers everywhere. We could not find a harness-side workaround. ## Critical path The most critical items for the milestone are [ruby/ruby#18534](https://github.com/ruby/ruby/pull/18534) #18534 (everything Zeitwerk-based, i.e. Rails, depends on it) and [ruby/ruby#18546](https://github.com/ruby/ruby/pull/18546) #18546 (Bundler's entire spec suite is gated behind it). After those, [ruby/ruby#18544](https://github.com/ruby/ruby/pull/18544) #18544 and [Bug #21867](https://bugs.ruby-lang.org/issues/21867) https://bugs.ruby-lang.org/issues/21867 let us drop most of the pending marks in the rubygems suite.