Bug #22397
closed
Memory corruption in rb_vm_insert_cc_refinement under compacting GC
Description
rb_vm_insert_cc_refinement() keeps a pointer into the embedded data of vm->cc_refinement_set across SIZED_REALLOC_N() of its entries. Under GC.stress, ruby_xrealloc2_sized() runs a GC before the realloc, and when that GC compacts and moves the set, the new buffer and capacity are stored in the abandoned slot while the moved set keeps the buffer the realloc has just freed.
The next growth then reallocates freed memory, which a RUBY_DEBUG build reports as [BUG] buffer 0x... realloced with old_size=2048, but was allocated with size=..., or the weak reference pass of the next GC walks the freed buffer and crashes. Call caches inserted after the move are also lost, so they are no longer invalidated.
It happened on the make (check, i686) job of https://github.com/ruby/ruby/pull/19133: https://github.com/ruby/ruby/actions/runs/36690728056/job/109807054948. TestEnumerator#test_with_index_under_gc_compact_stress ran after test_refinement.rb had filled the set, and the growth from 512 to 1024 entries moved the set. It needs the set to be on a page that compaction moves, so it is rare; forcing a compaction at that point in an instrumented build reproduces both crashes.
Updated by