Bug #19589: Expecting system stack error but crashing

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-04-11T00:54:20Z master 65e276096f) [arm64-darwin22]
• Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONE

[alpaca-tc (Hiroyuki Ishii)]

The following code expects stack overflow but crashes.
The version it occurs in is newer than 3.2.0.

def expect_system_stack_error(h)
  h.each_key {}
  h.each_value { expect_system_stack_error(h) }
end

expect_system_stack_error(a: nil)

[alpaca-tc (Hiroyuki Ishii)]

• Subject changed from Expecting stack overflow but crashing to Expecting system stack error but crashing

[jeremyevans0 (Jeremy Evans)]

• Backport changed from 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED

This happens when the iteration level in the hash goes from being stored in the flags to being stored in an instance variable (after 127 levels). Here is the top of the backtrace:

#0  rb_shape_get_iv_index (shape=0xb4b36a7afe0, id=13999, value=0x7f7ffffb9378) at shape.c:433                      
#1  0x00000b2be1b0ddd9 in generic_ivar_set (obj=12282921923520, id=13999, val=255) at variable.c:1271               
#2  ivar_set (obj=12282921923520, id=13999, val=255) at variable.c:1591                                             
#3  0x00000b2be1978f57 in hash_iter_lev_inc (hash=12282921923520) at hash.c:1367                                    
#4  rb_hash_foreach (hash=12282921923520, func=<optimized out>, farg=<optimized out>) at hash.c:1491                
#5  0x00000b2be197e4c0 in rb_hash_each_key (hash=12282921923520) at hash.c:3102                                     
#6  0x00000b2be1b3be8a in vm_call_cfunc_with_frame_ (ec=0xb2b5f97ae50, reg_cfp=0xb2c0ecb00c0,                       
    calling=<optimized out>, argc=0, argv=0xb2c0ebb8fa0, stack_bottom=0xb2c0ebb8f98) at ./vm_insnhelper.c:3434      
#7  0x00000b2be1b40252 in vm_sendish (ec=0xb2b5f97ae50, reg_cfp=0xb2c0ecb00c0, cd=0xb2c1144eea0,                    
    block_handler=<optimized out>, method_explorer=<optimized out>) at ./vm_insnhelper.c:5507                       
#8  0x00000b2be1b193f5 in vm_exec_core (ec=<optimized out>, initial=<optimized out>) at insns.def:815               
#9  0x00000b2be1b2ff54 in rb_vm_exec (ec=<optimized out>) at vm.c:2600                               

So this is likely related to the introduction of shapes.

[byroot (Jean Boussier)]

I have a fix for it: https://github.com/ruby/ruby/pull/7686

The flag operation in hash_iter_lev_dec was incorrect and could corrupt the object.

I think the bug is older than shapes introduction, but before it would corrupt unused memory. I still need to double check it doesn't have any other adverse effect on older rubies.

[byroot (Jean Boussier)]

• Status changed from Open to Closed

Applied in changeset git|8c360ce713f57d4177de833297364f6f6d950420.

---

hash.c: Fix hash_iter_lev_dec corrupting shape

[Bug #19589]

When decrementing iter_lev from 65 to 64 the flags would be
corrupted, causing the shape_id to be invalid.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONE

ruby_3_2 8165db0f4666f90fe0df5cdd466f73c6de1f6cea merged revision(s) 8c360ce713f57d4177de833297364f6f6d950420.