Bug #18795: Verbose GC debug output with -DRGENGC_DEBUG=5 causes a crash

• Status: Open
• Priority: Normal
• Assignee: -
• Target version: -
• ruby -v: ruby 3.2.0dev (2022-05-18T05:33:00Z master 97c12c5f69) [arm64-darwin21]
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

[kjtsanaktsidis (KJ Tsanaktsidis)]

I was trying to debug an (unrelated) issue in the GC, and wanted to turn on the trace-level GC output by compiling it with -DRGENGC_DEBUG=5. Unfortunately, this actually causes a crash in newobj_init() because the code there tries to log the obj_info() of the newly created object. However, the object is not actually sufficiently set up for some of the things that obj_info() tries to do:

• The instance variable table for a class is not yet initialized, and when using variable-length RVALUES, said ivar table is embedded in as-yet unitialized memory after the struct RValue. Attempting to read this, as obj_info() does, causes a crash.
• T_DATA variables need to dereference their ->type field to print out the underlying C type name, which is not set up until newobj_fill() is called.

I have attached a patch which fixes this by doing two things:

• Firstly, we define a new function obj_info_basic(), which is just obj_info() except skipping bits that don't work on newly-created objects. We make the new-object-creation log use this "basic" version instead of the ordinary obj_info().
• Then, move the log message out of newobj_init() and into newobj_fill(); all possible codepaths which call newobj_init() do subsequently call newobj_fill(), so this won't change any output, and it allows the type of T_DATA objects to still be printed even in obj_info_basic(), which is helpful.

Thanks!

[kjtsanaktsidis (KJ Tsanaktsidis)]

I had another think about this - the newobj tracepoint is also called in similar circumstances, where the ivar table for a new T_CLASS won't be initialized yet (if using variable length RVALUEs). Thus, calling rb_obj_memsize_of() on an object in its newobj tracepoint hook will crash.

I wonder if we should zero-initialize variable-length RValues?

[kjtsanaktsidis (KJ Tsanaktsidis)]

I opened a pull request with this patch: https://github.com/ruby/ruby/pull/5992