Bug #19387: ObjectSpace.each_objects only returns shareable objects after starting a Ractor

• Status: Closed
• Assignee: ractor
• Target version: -
• ruby -v: 3.3.0
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[luke-gru (Luke Gruber)]

r = Ractor.new do
  receive # block, the problem is not the termination of the ractor but the starting
end

ObjectSpace.each_object(IO) { |io|
  p io # we get no objects
}

[luke-gru (Luke Gruber)]

• Subject changed from Issue with ObjectSpace.each_objects not returning IO objects after starting a ractor to Issue with ObjectSpace.each_objects not returning objects after starting a ractor

The problem is actually for any objects, not just IO.

PR here: https://github.com/ruby/ruby/pull/7191

Edit: I took down the PR because I realized it doesn't properly fix the issue. The only way to get this working, and also to get ObjectSpace._id2ref working, is to keep the RACTOR_BELONGING_ID in all builds, and use that info when letting ractors access the objects.

I'll create another PR with that in mind. But I'm wondering why was this only in debug builds? Does it have to do with speed?

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned
• Assignee set to ko1 (Koichi Sasada)

• Status changed from Assigned to Closed

Applied in changeset git|c08fdc68383ee368c18e15e298502e6ee0089e18.

---

Allow waitpid(-1) to be woken if a waitpid(pid) call is pending

If two threads are running, with one calling waitpid(-1), and another
calling waitpid($some_pid), and then $some_other_pid exits, we would
expect the waitpid(-1) call to retrieve that exit status; however, it
cannot actually do so until $some_pid also exits.

This patch fixes the issue by unconditionally checking for pending
process group waits on SIGCHLD, and then allowing pending pid-only waits
to "steal" the notification.

[Fixes #19387]

[ko1 (Koichi Sasada)]

• Status changed from Closed to Assigned

c08fdc68383ee368c18e15e298502e6ee0089e18 is not related to this ticket.

• Status changed from Assigned to Closed

Applied in changeset git|0b7a4fbaa9c56d2c67d00d86c69f9e5c71803267.

---

Allow waitpid(-1) to be woken if a waitpid(pid) call is pending

If two threads are running, with one calling waitpid(-1), and another
calling waitpid($some_pid), and then $some_other_pid exits, we would
expect the waitpid(-1) call to retrieve that exit status; however, it
cannot actually do so until $some_pid also exits.

This patch fixes the issue by unconditionally checking for pending
process group waits on SIGCHLD, and then allowing pending pid-only waits
to "steal" the notification.

[Fixes #19387]

[luke-gru (Luke Gruber)]

• ruby -v set to 3.3.0

I believe this was closed prematurely due to an unrelated git commit that wrongly tagged it. Please reopen :)

[jeremyevans0 (Jeremy Evans)]

• Status changed from Closed to Open

• Status changed from Open to Closed

Applied in changeset git|76a8c963c7ad975b7bbfc1c4979bf7a2de15af27.

---

Add a test for what happens with concurent calls to waitpid

Ruby 3.1 and 3.2 have a bug in their implementation, for which I'm
backporting a fix. However, the current development branch doesn't have
the issue (because the MJIT -> RJIT change refactored how waitpid worked
substantially). I do however want to commit the test which verifies
that waitpid works properly on master.

[Fixes #19387]

[luke-gru (Luke Gruber)]

It happened again :( I notified the committer of the mistake. Please reopen, thanks!

[jeremyevans0 (Jeremy Evans)]

• Status changed from Closed to Open

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned

[wanabe (_ wanabe)]

• Has duplicate Bug #21149: Strange behavior of ObjectSpace.each_object after Ractor.new added

[Eregon (Benoit Daloze)]

One idea to solve this would be to implement ObjectSpace.each_object like TruffleRuby does it (at least when there are multiple Ractors):
basically it's just the transitive ObjectSpace.reachable_objects_from(roots).
I.e. it uses the same mechanism as GC traversal/marking to find every object reachable from some roots objects (like Object class, the stack frames, etc).

Context: the latest breakage due to this bug: https://github.com/ruby/prism/pull/3491
And I know of many before, it's a very common issue that Ractor.new can't be used in make test-all.

[tenderlovemaking (Aaron Patterson)]

• Assignee changed from ko1 (Koichi Sasada) to ractor

[osyoyu (Daisuke Aritomo)]

I also ran into this, so I opened an pull request document this behavior.
https://github.com/ruby/ruby/pull/13278

[Eregon (Benoit Daloze)]

• Related to Feature #17270: ObjectSpace.each_object should be restricted on multi-Ractors added

[jhawthorn (John Hawthorn)]

• Has duplicate Bug #21401: ObjectSpace can't count Fibers after using a Ractor added

[jhawthorn (John Hawthorn)]

• Subject changed from Issue with ObjectSpace.each_objects not returning objects after starting a ractor to ObjectSpace.each_objects only returns shareable objects after starting a Ractor

[ko1 (Koichi Sasada)]

• Status changed from Assigned to Closed

Re-checked on master 4d185cce78: this no longer reproduces.

With per-Ractor GC (GH-18194) each Ractor has its own objspace, and
ObjectSpace.each_object walks the calling Ractor's objspace directly, so it keeps
working while other Ractors are alive:

port = Ractor::Port.new
r = Ractor.new(port) { |p| p << :up; Ractor.receive }
port.receive
p ObjectSpace.each_object(IO).count

ruby 4.0.2 : 0
master     : 3

Isolation is still respected. Letting a sub-Ractor build one unshareable and one
shareable String and scanning for them from the main Ractor:

                                      4.0.2    master
own unshareable objects                no       yes
another Ractor's unshareable objects   no       no
another Ractor's shareable objects     yes      yes

(the last row needs the object to have actually been made shareable; a merely frozen
String that nobody has shared yet does not carry the flag and is not enumerated.)

So the reported symptom -- each_object yields nothing once a Ractor has been started
-- is gone, and Ractor.new can be used in make test-all again.

One follow-up: the documentation added in GH-13278 ("Due to a current Ractor
implementation issue, this method does not yield Ractor-unshareable objects when the
process is in multi-Ractor mode", which links to this ticket) is now wrong. I will send
a separate PR for it.