Bug #22467
closedIO#readpartial can raise EFAULT during GC.compact in another thread
Description
This was observed in a test suite that calls GC.compact before forking. A thread doing socket.readpartial(512) in a loop died with Errno::EFAULT: Bad address, in ~10% of runs. After dropping the GC.compact, the error was no longer produced.
Concise reproduction script (usually fails within 10 compactions):
# Makes IO#readpartial raise Errno::EFAULT when GC.compact runs in another thread.
require 'socket'
socket, peer = UNIXSocket.pair
delays, delays_in = IO.pipe
# The data must come from another process: no thread in this one runs while it compacts.
writer = fork { loop { sleep(delays.read(1).ord / 1000.0); peer.write('x') } }
buffers = Queue.new
reader = Thread.new { loop { socket.readpartial(512, buffers.pop) } }
reader.report_on_exception = false
begin
100.times do
# Sparse pages of 640-byte slots with one pinned object each (identity hash keys are pinned).
# Compaction empties and locks such pages; the buffer is one of the pinned objects.
strings = Array.new(200_000) { 'y' * 500 }
pinned = {}.compare_by_identity
movable = strings.each_slice(20).map {|slice| pinned[slice[0]] = true; slice[10] }
buffers << strings[100_000]
strings = nil
sleep 0.01 # the reader drains the socket and waits in poll(2)
delays_in.write(rand(30).chr) # the next byte arrives somewhere during the compaction
GC.compact
reader.join(0)
end
puts "#{RUBY_DESCRIPTION}: no error"
rescue => e
puts "#{RUBY_DESCRIPTION}: #{e.class}: #{e.message}"
ensure
Process.kill(:KILL, writer)
end
The following explanation by Claude is outside of my expertise to verify:
This is the read-side counterpart of #20169. The read path still passes RSTRING_PTR(str) of a possibly embedded string to read(2) without the GVL. If another thread is running GC.compact and has locked that string's heap page with mprotect(PROT_NONE), the kernel cannot copy the data and read(2) fails with EFAULT.
In io_getpartial, iis.buf = RSTRING_PTR(str) is passed to io_read_memory_locktmp. internal_read_func then runs read(2), and on EAGAIN it waits in poll(2) and retries read(2), all inside the blocking region. So if the socket is idle when compaction starts and data arrives while the buffer's page is locked, the read fails.
... sysread, read_nonblock, write_nonblock and syswrite look like they also pass the string's memory to a syscall without the GVL, so they are probably affected too.
Updated by peterzhu2118 (Peter Zhu) 26 days ago
- Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: WONTFIX, 3.4: REQUIRED, 4.0: REQUIRED
Thank you for the bug report, I have a fix here: https://github.com/ruby/ruby/pull/19515
Updated by peterzhu2118 (Peter Zhu) 25 days ago
- Status changed from Open to Closed
Applied in changeset git|0b7838e23b8f9b01765dafd5e675da6113f5b566.
Fix IO reading without GVL during GC compaction
[Bug #22467]
If we perform IO read without the GVL during GC compaction and the string
we are writing to is embedded, then it can get mprotect'd by the read
barrier in the GC, which can cause the IO read operation to fail.
Updated by nagachika (Tomoyuki Chikanaga) 11 days ago
- Backport changed from 3.3: WONTFIX, 3.4: REQUIRED, 4.0: REQUIRED to 3.3: WONTFIX, 3.4: DONE, 4.0: REQUIRED
ruby_3_4 340a3a8b249bbb735fa1169941daeeadd084811f merged revision(s) 8094e4e2477fc68577bd36447424a61e75749992, 0b7838e23b8f9b01765dafd5e675da6113f5b566.
Updated by nagachika (Tomoyuki Chikanaga) 7 days ago
It appears that e131720215251851efab1811a24c4d59a0fc979d [Bug #22389] is also required to fix test hang-ups on macOS.
I have filled REQUIRED in the Backport field.
Updated by k0kubun (Takashi Kokubun) 3 days ago
- Backport changed from 3.3: WONTFIX, 3.4: DONE, 4.0: REQUIRED to 3.3: WONTFIX, 3.4: DONE, 4.0: DONE
ruby_4_0 c26506e88a95423dd252634a61174c5b613ec283 merged revision(s) 0b7838e23b8f9b01765dafd5e675da6113f5b566.