Bug #22397: Memory corruption in rb_vm_insert_cc_refinement under compacting GC

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 4.1.0dev (2026-09-30 master 5c128f545b)
• Backport: 3.3: DONTNEED, 3.4: DONTNEED, 4.0: DONTNEED

[shugo (Shugo Maeda)]

rb_vm_insert_cc_refinement() keeps a pointer into the embedded data of vm->cc_refinement_set across SIZED_REALLOC_N() of its entries. Under GC.stress, ruby_xrealloc2_sized() runs a GC before the realloc, and when that GC compacts and moves the set, the new buffer and capacity are stored in the abandoned slot while the moved set keeps the buffer the realloc has just freed.

The next growth then reallocates freed memory, which a RUBY_DEBUG build reports as [BUG] buffer 0x... realloced with old_size=2048, but was allocated with size=..., or the weak reference pass of the next GC walks the freed buffer and crashes. Call caches inserted after the move are also lost, so they are no longer invalidated.

It happened on the make (check, i686) job of https://github.com/ruby/ruby/pull/19133: https://github.com/ruby/ruby/actions/runs/36690728056/job/109807054948. TestEnumerator#test_with_index_under_gc_compact_stress ran after test_refinement.rb had filled the set, and the growth from 512 to 1024 entries moved the set. It needs the set to be on a page that compaction moves, so it is rare; forcing a compaction at that point in an instrumented build reproduces both crashes.

[shugo (Shugo Maeda)]

• Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: DONTNEED, 3.4: DONTNEED, 4.0: DONTNEED

[shugo (Shugo Maeda)]

• Status changed from Open to Closed

Applied in changeset git|dd7be7ce695a118104c132c5aec917884fc1974b.

---

Re-read the refinement call cache set after growing it

rb_vm_insert_cc_refinement() kept a pointer into the embedded data of
vm->cc_refinement_set across the realloc of its entries. Under
GC.stress, ruby_xrealloc2_sized() runs a GC first, and when that GC
compacts and moves the set, the new buffer and capacity are written to
the abandoned slot while the moved set keeps the freed buffer. The next
growth then reallocates freed memory, or the weak reference pass of the
next GC walks it.

[Bug #22397]

Co-Authored-By: Claude Fable 5.1 [e-mail]