Bug #22435: Autoload under a Fiber scheduler causes a deadlock or [BUG]

• Status: Assigned
• Assignee: ioquatix (Samuel Williams)
• Target version: -
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

[mame (Yusuke Endoh)]

The following code causes a deadlock. test/fiber/scheduler.rb is the one in the ruby/ruby repository (https://github.com/ruby/ruby/blob/master/test/fiber/scheduler.rb).

# fs-autoload-deadlock.rb
require_relative "test/fiber/scheduler"

class MyScheduler < Scheduler
  def blocking_operation_wait(work)
    @workers ||= Fiber.blocking { Etc.nprocessors } # lazily size a worker pool
    super
  end
end

autoload :Etc, "etc"
autoload :JSON, "json"
$LOADED_FEATURES << "dummy"  # force to rebuild the index of $LOADED_FEATURES

Thread.new do
  Fiber.set_scheduler(MyScheduler.new)
  Fiber.schedule { p JSON }
end.join

$ ruby fs-autoload-deadlock.rb
#<Thread:0x00007ce4cf1eed90 fs-autoload-deadlock.rb:14 run> terminated with exception (report_on_exception is true):
fs-autoload-deadlock.rb:5:in 'block in MyScheduler#blocking_operation_wait': deadlock; recursive locking (ThreadError)
	from fs-autoload-deadlock.rb:5:in 'Fiber.blocking'
	from fs-autoload-deadlock.rb:5:in 'MyScheduler#blocking_operation_wait'
	from fs-autoload-deadlock.rb:16:in 'block (2 levels) in <main>'
...

Also, the following code causes [BUG].

# fs-autoload-segv.rb
require_relative "test/fiber/scheduler"

class MyScheduler < Scheduler
  def block(blocker, timeout = nil)
    blocker.inspect  #=> [BUG] Segmentation fault at 0x0000000000000000
    super
  end
end

autoload :JSON, "json"
$LOADED_FEATURES << "dummy"  # force to rebuild the index of $LOADED_FEATURES

Thread.new do
  Fiber.set_scheduler(MyScheduler.new)
  Fiber.schedule { JSON }
  Fiber.schedule { JSON }
end.join

$ ruby fs-autoload-segv.rb
fs-autoload-segv.rb:5: [BUG] Segmentation fault at 0x0000000000000000
ruby 4.1.0dev (2026-08-21T07:50:27Z master c9e9101403) +PRISM [x86_64-linux]
...
-- Ruby level backtrace information ----------------------------------------
fs-autoload-segv.rb:16:in 'block (2 levels) in <main>'
fs-autoload-segv.rb:5:in 'block'
...

Both reproduce on master and 4.0.6.

$LOADED_FEATURES << "dummy" is to make Ruby rebuild the index of $LOADED_FEATURES (stat(2) during the rebuild calls blocking_operation_wait).

Cause

The cause is the autoload mutex introduced by 32de6097b2b5d8394b3a1399e13d309444697954 (https://github.com/ruby/ruby/pull/5898).
The deadlock above occurs because this mutex is locked recursively.
Also, when a fiber blocks on locking the mutex, the mutex is passed to Fiber::Scheduler#block as the blocker, but it is a hidden object, which leads to rb_bug.

Analysis

Let me recap the problem that 32de6097b2 tried to solve, [Bug #18782].

Autoload roughly proceeded in the following steps 1, 2, and 3.

1. If the mutex to synchronize requiring the feature has already been created and another thread is locking it, wait, because that thread is requiring the feature
2. If the feature has not been required yet, create a mutex for the feature, lock it, and require the feature
3. Refer to the constant that should have been defined

Consider the following code.

autoload :Foo, "foobar.rb"
autoload :Bar, "foobar.rb"

tA = Thread.new {Foo}
tB = Thread.new {Bar}

If tA and tB executed the steps 1, 2, and 3 in the following order, a NameError was raised in the thread tA.

• tA: executes step 1 (there is no mutex, so it proceeds without waiting)
• context switch
• tB: executes step 1 (there is no mutex, so it proceeds without waiting)
• tB: executes step 2 (creates and locks the mutex, and requires the feature. When the require finishes, the feature is marked as required, but the autoload constants are defined after that)
• context switch
• tA: executes step 2 (the feature has been required, so it does nothing)
• tA: executes step 3 (the constant is not defined yet, so NameError)

The check in step 2, whether the feature has not been required yet, may call to_str of an element of $LOADED_FEATURES or release the GVL, so a context switch can occur there.

Solution

32de6097b2 tried to solve it by adding a new mutex to lock the steps 1, 2, and 3 of autoload, but as shown above, this is a new source of deadlocks and adds complexity, so I don't think it is good.

I think it can be solved simply by swapping the steps 1 and 2 of autoload.

1'. If the feature has not been required yet, create a mutex for the feature, lock it, and require the feature
2'. If the mutex to synchronize requiring the feature has already been created and another thread is locking it, wait, because that thread is requiring the feature
3. Refer to the constant that should have been defined

The thread that requires the feature keeps the mutex locked until it finishes defining the constants. So, if the feature looks "required" at 1' but the constants are not defined yet, the mutex is still locked at 2', and the thread waits for it.

Also, even if both threads see the feature as not required yet at 1', only one mutex is created. Checking whether the mutex exists and creating it involve no Ruby code nor GVL release, so no thread switch occurs between them thanks to the GVL. The thread that comes later finds the mutex created by the other, and waits for it.

This solves the NameError problem of [Bug #18782] without the global autoload mutex.

So, how about removing the global autoload mutex introduced by 32de6097b2, and swapping the steps as above?

PR: https://github.com/ruby/ruby/pull/19335

In addition to passing all Ruby tests, I also checked the tests of zeitwerk and bootsnap just in case. All tests of zeitwerk pass. One test of bootsnap (test_load_static_libaries) fails, but it is because prism is no longer a default gem on master, and it is unrelated to this change.