Bug #19419: [BUG] try to mark T_NONE object in `ibf_dump_mark`

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.2.0 (2022-12-25 revision a528908271)
• Backport: 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: DONE

[byroot (Jean Boussier)]

Reported downstream at https://github.com/Shopify/bootsnap/issues/436

Relevant part of the crash report:

[BUG] try to mark T_NONE object

-- C level backtrace information -------------------------------------------
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_print_backtrace+0xd) [0x7fb6b050ab39] ruby-3.2.0/vm_dump.c:785
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_vm_bugreport) ruby-3.2.0/vm_dump.c:1080
Ruby/3.2.0/x64/lib/libruby.so.3.2(bug_report_end+0x0) [0x7fb6b0301049] ruby-3.2.0/error.c:790
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_bug_without_die) ruby-3.2.0/error.c:790
Ruby/3.2.0/x64/lib/libruby.so.3.2(die+0x0) [0x7fb6b025ae4f] ruby-3.2.0/error.c:798
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_bug) ruby-3.2.0/error.c:800
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_mark_ptr+0xcf) [0x7fb6b032a43f] ruby-3.2.0/gc.c:7059
Ruby/3.2.0/x64/lib/libruby.so.3.2(mark_key+0x2f) [0x7fb6b032a75f] ruby-3.2.0/gc.c:7085
Ruby/3.2.0/x64/lib/libruby.so.3.2(apply_functor+0x13) [0x7fb6b0466bc5] ruby-3.2.0/st.c:1574
Ruby/3.2.0/x64/lib/libruby.so.3.2(st_general_foreach) ruby-3.2.0/st.c:1484
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_st_foreach) ruby-3.2.0/st.c:1581
Ruby/3.2.0/x64/lib/libruby.so.3.2(ibf_dump_mark+0x1a) [0x7fb6b029895a] ruby-3.2.0/compile.c:13033
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_mark_stacked_objects+0x78) [0x7fb6b032d628] ruby-3.2.0/gc.c:7437
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_mark_stacked_objects_all) ruby-3.2.0/gc.c:7477
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_marks_rest) ruby-3.2.0/gc.c:8675
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_marks+0x423) [0x7fb6b032e643] ruby-3.2.0/gc.c:8716
Ruby/3.2.0/x64/lib/libruby.so.3.2(gc_start) ruby-3.2.0/gc.c:9547
Ruby/3.2.0/x64/lib/libruby.so.3.2(heap_prepare+0x2d) [0x7fb6b03319c8] ruby-3.2.0/gc.c:2431
Ruby/3.2.0/x64/lib/libruby.so.3.2(heap_next_free_page) ruby-3.2.0/gc.c:2672
Ruby/3.2.0/x64/lib/libruby.so.3.2(newobj_alloc) ruby-3.2.0/gc.c:2780
Ruby/3.2.0/x64/lib/libruby.so.3.2(newobj_of0+0x66) [0x7fb6b0332441] ruby-3.2.0/gc.c:2876
Ruby/3.2.0/x64/lib/libruby.so.3.2(newobj_of) ruby-3.2.0/gc.c:2896
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_wb_protected_newobj_of) ruby-3.2.0/gc.c:2918
Ruby/3.2.0/x64/lib/libruby.so.3.2(str_alloc_embed+0x10) [0x7fb6b046f9a8] ruby-3.2.0/string.c:890
Ruby/3.2.0/x64/lib/libruby.so.3.2(str_new0) ruby-3.2.0/string.c:926
Ruby/3.2.0/x64/lib/libruby.so.3.2(str_new_frozen_buffer+0x7d) [0x7fb6b04757ed] ruby-3.2.0/string.c:1461
Ruby/3.2.0/x64/lib/libruby.so.3.2(rb_file_directory_p+0xf4) [0x7fb6b03114d4] ruby-3.2.0/file.c:1309

Unfortunately like many GC bugs, we don't have a reproduction, the initial reporter isn't able to reproduce it reliably.

[byroot (Jean Boussier)]

• Description updated (diff)

[byroot (Jean Boussier)]

• Backport changed from 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN to 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED

The original bug reporter provided a relatively simple reproduction repository: https://github.com/tisba/bootsnap-ruby-crash

Using it I was able to reproduce on 3.2.1 and latest master.

I also confirmed that it's a Write Barrier bug, as removing the WB_PROTECTED flag in ibf_dump_type does make the issue disappear.

[alanwu (Alan Wu)]

• Backport changed from 2.7: DONTNEED, 3.0: DONTNEED, 3.1: DONTNEED, 3.2: REQUIRED to 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED

It turned out to be a write barrier issue: https://github.com/ruby/ruby/pull/7296
The same problem issue exists on 3.1.x, so marking for backport.

[alanwu (Alan Wu)]

• Backport changed from 2.7: DONTNEED, 3.0: DONTNEED, 3.1: REQUIRED, 3.2: REQUIRED to 2.7: DONTNEED, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED

[alanwu (Alan Wu)]

• Status changed from Open to Closed

Applied in changeset git|86de48e9f69b665ba9ffb5bdc5a181a3adb1a7b8.

---

Remove ibf_dumper's WB_PROTECTED status

It doesn't have the right write barriers in place. For example, there is

rb_mark_set(dump->global_buffer.obj_table);

in the mark function, but there is no corresponding write barrier when
adding to the table in the
ibf_dump_object() -> ibf_table_find_or_insert() -> st_insert() code path.

To insert write barrier correctly, we need to store the T_STRUCT VALUE
inside struct ibf_dump. Instead of doing that, let's just demote it
to WB unproected for correctness. These dumper object are ephemeral so
there is not a huge benefit for having them WB protected.

Users of the bootsnap gem ran into crashes due to this issue:
https://github.com/Shopify/bootsnap/issues/436

Fixes [Bug #19419]

[naruse (Yui NARUSE)]

• Backport changed from 2.7: DONTNEED, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: REQUIRED to 2.7: DONTNEED, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONE

ruby_3_2 f1cde05d99898f491c8e302ae74029468fdb6eb9 merged revision(s) 86de48e9f69b665ba9ffb5bdc5a181a3adb1a7b8.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.7: DONTNEED, 3.0: REQUIRED, 3.1: REQUIRED, 3.2: DONE to 2.7: DONTNEED, 3.0: REQUIRED, 3.1: DONE, 3.2: DONE

ruby_3_1 caedcf3ee8e445f90df88ccf2f745c9d9f7ccc35 merged revision(s) 86de48e9f69b665ba9ffb5bdc5a181a3adb1a7b8.