Bug #22305: Ruby::Box: statically linked prism loads in only one box

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 4.1.0dev (2026-09-10T03:20:50Z master fe58143f12) +YJIT +MN +PRISM [arm64-darwin27]
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

[hsbt (Hiroshi SHIBATA)]

Under RUBY_BOX=1, the statically linked prism can be required in only one box. The second box raises LoadError.

$ RUBY_BOX=1 ruby --disable-gems -e 'require "prism"; p Ruby::Box.new.eval(%q{require "prism"; Prism::VERSION})'
lib/prism.rb:161:in 'Kernel#require': cannot load such file -- prism/prism (LoadError)

Swapping the order fails the same way. --disable-gems matters here. With a prism gem installed, each box dlopens its own copy of the gem's prism.bundle and the problem disappears, which makes it easy to miss.

There is no prism.so file. prism_init.c:8 registers the name prism/prism.so with ruby_init_ext, pointing at the Init_prism already linked into the interpreter, so require "prism/prism" behaves as if it had loaded an extension. That entry can be used only once (load.c:1280). The parser itself works in every box, but loading a gem that requires prism internally, such as error_highlight, is enough to break it in every box other than the one that loaded prism first.

The obvious fixes do not hold. Running the init in every box leaves one set of C globals pointing at whichever box ran it last, so nodes parsed in the other box fail is_a?(Prism::ProgramNode). That trades the LoadError for a silent type mismatch. Initializing prism before the boxes exist and marking the feature provided makes the require "prism/prism" inside an activated prism gem a no-op, so the gem's Ruby half binds to the interpreter's C half. The ruby-lsp benchmark fails on exactly that.

I see three designs and would like one chosen.

The first treats prism as an extension. The Ruby binding is built as its own shared object while the parser stays static, so each box dlopens its own copy the way it already does for other extensions. The same principle would also fix every extension under --with-static-linked-ext. It changes the build and needs symbol exports on Windows.

The second treats it as part of the interpreter, like String. It is initialized once before the boxes and every box shares its classes. defined?(Prism) becomes true without a require, it adds 339 classes at boot, and a prism gem can no longer take over.

The third changes Box instead. One way would be to share a statically linked extension across boxes like a builtin class without marking it provided, so that a prism gem can still take over, though I am not sure that fits where Box is heading.

How prism came to ship this way

#19741 mirrored YARP into ruby/ruby together with its Ruby API and registered the native part with ruby_init_ext. #19772 then moved toward mirroring only the C implementation and leaving the Ruby API to a bundled prism gem (#19772#note-29), and #19900 stated that prism would be a bundled gem with a native extension. The static registration from #19741 survived the rename, so prism ships as a default gem linked into the interpreter, and that registration is what fails here. The native extension #19900 described works in every box, and the first design comes down to it.

The second design has a history of its own. Matz first approved a built-in Ruby::Parser that no gem could replace (#19772#note-9), and the same idea came back as a gem-independent Ruby::Node for Proc#syntax_tree (#21795#note-1). There Matz required that the parser that ran the program and the parser that returns the tree be the same (#21795#note-19), and the feature settled on the default gem prism for that. That is exactly the prism a second box cannot load, which also blocks the way #21795#note-38 suggested Box could eventually expose the interpreter's own parser.

[hsbt (Hiroshi SHIBATA)]

• Related to Misc #22275: Ruby::Box support plan for RubyGems and Bundler added

[hsbt (Hiroshi SHIBATA)]

• Related to Feature #19741: Mirror YARP to Ruby added

[hsbt (Hiroshi SHIBATA)]

• Related to Misc #19772: API naming for YARP compiler added

[hsbt (Hiroshi SHIBATA)]

• Related to Feature #19900: Rename YARP to Prism added

[hsbt (Hiroshi SHIBATA)]

• Related to Bug #21618: Allow to use the build-in prism version to parse code added

[hsbt (Hiroshi SHIBATA)]

• Related to Feature #21795: Methods for retrieving ASTs added