Verbose GC debug output with -DRGENGC_DEBUG=5 causes a crash
I was trying to debug an (unrelated) issue in the GC, and wanted to turn on the trace-level GC output by compiling it with -DRGENGC_DEBUG=5. Unfortunately, this actually causes a crash in newobj_init() because the code there tries to log the obj_info() of the newly created object. However, the object is not actually sufficiently set up for some of the things that obj_info() tries to do:
- The instance variable table for a class is not yet initialized, and when using variable-length RVALUES, said ivar table is embedded in as-yet unitialized memory after the struct RValue. Attempting to read this, as obj_info() does, causes a crash.
- T_DATA variables need to dereference their ->type field to print out the underlying C type name, which is not set up until newobj_fill() is called.
I have attached a patch which fixes this by doing two things:
- Firstly, we define a new function obj_info_basic(), which is just obj_info() except skipping bits that don't work on newly-created objects. We make the new-object-creation log use this "basic" version instead of the ordinary obj_info().
- Then, move the log message out of newobj_init() and into newobj_fill(); all possible codepaths which call newobj_init() do subsequently call newobj_fill(), so this won't change any output, and it allows the type of T_DATA objects to still be printed even in obj_info_basic(), which is helpful.
Updated by kjtsanaktsidis (KJ Tsanaktsidis) 3 months ago
I had another think about this - the newobj tracepoint is also called in similar circumstances, where the ivar table for a new T_CLASS won't be initialized yet (if using variable length RVALUEs). Thus, calling rb_obj_memsize_of() on an object in its newobj tracepoint hook will crash.
I wonder if we should zero-initialize variable-length RValues?