Bug #20501: ruby SEGV

• Status: Closed
• Assignee: tenderlovemaking (Aaron Patterson)
• Target version: -
• Backport: 3.1: WONTFIX, 3.2: WONTFIX, 3.3: REQUIRED

[akr (Akira Tanaka)]

I encountered SEGV of ruby.

% ./ruby -v
ruby 3.4.0dev (2024-05-22T06:59:19Z master 5613d6e95b) [x86_64-linux]
% ./ruby t.rb
t.rb:33: [BUG] Segmentation fault at 0x00007fc243696098
ruby 3.4.0dev (2024-05-22T06:59:19Z master 5613d6e95b) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0003 p:0000 s:4294967313 e:000018 METHOD t.rb:33
c:0002 p:0022 s:0006 e:000005 EVAL   t.rb:52 [FINISH]
c:0001 p:0000 s:0003 E:000350 DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
t.rb:52:in '<main>'
t.rb:33:in 'create_no_file'

-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- Machine register context ------------------------------------------------
 RIP: 0x000055a1cdc8bb9c RBP: 0x000055a1cee844b0 RSP: 0x00007ffcde5cdae0
 RAX: 0x00007fc2436960a0 RBX: 0x00007fba43795f68 RCX: 0x0000000000000000
 RDX: 0x000055a1cf115cf0 RDI: 0x0000000000000009 RSI: 0x00007fba28526860
  R8: 0x00007fba436960a1  R9: 0x0000000000000000 R10: 0x00007fba28526860
 R11: 0x0000000000000003 R12: 0x0000000000000006 R13: 0x00007fba2853b698
 R14: 0x0000000d00000009 R15: 0x0000000000000b21 EFL: 0x0000000000010246

-- C level backtrace information -------------------------------------------
/home/ruby/t2/ruby/ruby(rb_print_backtrace+0x14) [0x55a1cdcae243] /home/ruby/t2/ruby/vm_dump.c:820
/home/ruby/t2/ruby/ruby(rb_vm_bugreport) /home/ruby/t2/ruby/vm_dump.c:1151
/home/ruby/t2/ruby/ruby(rb_bug_for_fatal_signal+0xf8) [0x55a1cde5abe8] /home/ruby/t2/ruby/error.c:1108
/home/ruby/t2/ruby/ruby(sigsegv+0x44) [0x55a1cdbf7864] /home/ruby/t2/ruby/signal.c:929
/lib/x86_64-linux-gnu/libc.so.6(0x7fba438f8050) [0x7fba438f8050]
/home/ruby/t2/ruby/ruby(vm_exec_handle_exception+0x2ac) [0x55a1cdc8bb9c] /home/ruby/t2/ruby/vm.c:2782
...

t.rb and the full crash report are attached.

[akr (Akira Tanaka)]

t.rb is not minimized because the probability of SEGV is reduced when I make the file smaller.

[nobu (Nobuyoshi Nakada)]

Although I can't reproduce it, it looks like that catch_iseq is broken, from the backtrace.

[akr (Akira Tanaka)]

• File crash2.txt crash2.txt added

git bisect shows the problem is caused by the following commit.

% git bisect bad 
84e4453436c3549b4fda6014cdd5fcc9e0b80755 is the first bad commit
commit 84e4453436c3549b4fda6014cdd5fcc9e0b80755
Author: Aaron Patterson <[e-mail]>
Date:   Tue Feb 7 17:46:42 2023 -0800

    Use a functional red-black tree for indexing the shapes
    
    This is an experimental commit that uses a functional red-black tree to
    create an index of the ancestor shapes.  It uses an Okasaki style
    functional red black tree:
    
      https://www.cs.tufts.edu/comp/150FP/archive/chris-okasaki/redblack99.pdf
    
    This tree is advantageous because:
    
    * It offers O(n log n) insertions and O(n log n) lookups.
    * It shares memory with previous "versions" of the tree
    
    When we insert a node in the tree, only the parts of the tree that need
    to be rebalanced are newly allocated.  Parts of the tree that don't need
    to be rebalanced are not reallocated, so "new trees" are able to share
    memory with old trees.  This is in contrast to a sorted set where we
    would have to duplicate the set, and also resort the set on each
    insertion.
    
    I've added a new stat to RubyVM.stat so we can understand how the red
    black tree increases.

 benchmark/vm_ivar_ic_miss.yml |  20 +++
 rjit_c.rb                     |   5 +
 shape.c                       | 309 ++++++++++++++++++++++++++++++++++++++++--
 shape.h                       |  15 ++
 vm.c                          |   8 +-
 5 files changed, 342 insertions(+), 15 deletions(-)
 create mode 100644 benchmark/vm_ivar_ic_miss.yml

% ./miniruby -v
ruby 3.3.0dev (2023-10-24T17:52:06Z v3_3_0_preview3~561 84e4453436) [x86_64-linux]
% ./miniruby t.rb
t.rb:32593: [BUG] Segmentation fault at 0x00007f596f3e1098
ruby 3.3.0dev (2023-10-24T17:52:06Z v3_3_0_preview3~561 84e4453436) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0003 p:21907 s:4294967313 e:000018 METHOD t.rb:32593
c:0002 p:0022 s:0006 e:000005 EVAL   t.rb:52 [FINISH]
c:0001 p:0000 s:0003 E:001b90 DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
t.rb:52:in `<main>'
t.rb:32593:in `create_no_file'

-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- Machine register context ------------------------------------------------
 RIP: 0x0000559323b21d31 RBP: 0x00007ffd7d3f5730 RSP: 0x00007ffd7d3f55e0
 RAX: 0x00007f596f3e1098 RBX: 0x00007f51542a27c0 RCX: 0x00007f516f4e0f30
 RDX: 0x00007f51542a27c0 RDI: 0x00007f516f3e10a0 RSI: 0x0000000000000080
  R8: 0x00000000000005f5  R9: 0x0000559325ad6220 R10: 0x06c15729dbba7803
 R11: 0x0000000000000090 R12: 0x00007f516f3e1040 R13: 0x0000000000000000
 R14: 0x000055932591c490 R15: 0x00007f516f4e0f30 EFL: 0x0000000000010206

-- C level backtrace information -------------------------------------------
/home/ruby/t4/ruby/miniruby(rb_print_backtrace+0x20) [0x559323b2be67] /home/ruby/t4/ruby/vm_dump.c:812
/home/ruby/t4/ruby/miniruby(rb_vm_bugreport+0x28c) [0x559323b2c57e] /home/ruby/t4/ruby/vm_dump.c:1143
./miniruby(rb_bug_for_fatal_signal+0x143) [0x559323906dd1]
/home/ruby/t4/ruby/miniruby(sigsegv+0x75) [0x559323a70ced] /home/ruby/t4/ruby/signal.c:920
/home/ruby/t4/ruby/miniruby(sigill) (null):0
/lib/x86_64-linux-gnu/libc.so.6(0x7f516f641050) [0x7f516f641050]
/home/ruby/t4/ruby/miniruby(vm_exec_handle_exception+0xb18) [0x559323b21d31] /home/ruby/t4/ruby/vm.c:2687
...

The full crash log is attached as crash2.txt.

[mame (Yusuke Endoh)]

• Status changed from Open to Assigned
• Assignee set to tenderlovemaking (Aaron Patterson)

[luke-gru (Luke Gruber)]

It looks like it has to do with defined?() in an if expression and its catch table entries when the first part of the if expression has been eliminated and there is no then label to follow.

For example, this is a minimal reproduction:

def my_method
  ivar = nil # seems to be needed to reproduce the issue
  if false && defined? File::TMPFILE
  end
  raise "woops"
end
my_method

The defined catch table entry is not setup correctly in this case.

I only tried this on ruby head, but I get a segfault.

[luke-gru (Luke Gruber)]

I created a patch here: https://github.com/ruby/ruby/pull/11554. I'm new to the code in compile.c so perhaps someone could come up with a better solution.

[nobu (Nobuyoshi Nakada)]

luke-gru (Luke Gruber) wrote in #note-5:

For example, this is a minimal reproduction:

I can't reproduce it with master.

def my_method
  ivar = nil # seems to be needed to reproduce the issue
  if false && defined? File::TMPFILE
  end
  raise "woops"
end
my_method

$ ./bin/ruby -v bug-20501.rb 
ruby 3.4.0dev (2024-09-06T00:32:47Z master 81b74c9fad) [aarch64-linux]
bug-20501.rb:2: warning: assigned but unused variable - ivar
bug-20501.rb:5:in 'Object#my_method': woops (RuntimeError)
	from bug-20501.rb:7:in '<main>'

$ ./ruby -v bug-20501.rb 
ruby 3.4.0dev (2024-09-06T00:32:47Z master 81b74c9fad) [arm64-darwin23]
bug-20501.rb:2: warning: assigned but unused variable - ivar
bug-20501.rb:5:in 'Object#my_method': woops (RuntimeError)
	from bug-20501.rb:7:in '<main>'

[luke-gru (Luke Gruber)]

I'm on x86-64 linux so that might have to do with it. I'll investigate a bit more.

[mdalessio (Mike Dalessio)]

I'm not able to reproduce on Linux with this script using either master HEAD or 5613d6e95b.

[luke-gru (Luke Gruber)]

This is a weird way to reproduce, but you can see it on https://runruby.dev/ if you comment out the Gemfile and put this in main.rb:

def my_method
  var = nil
  if false && defined? File::TMPFILE
  end
  raise "woops"
end
puts RubyVM::InstructionSequence.disasm(method(:my_method)) # you should see the invalid catch table entry in the disasm bytecode
my_method() # if this doesn't trigger an error, try running it multiple times.

[luke-gru (Luke Gruber)]

Okay, I figured out what's happening. In compile.c, new LABELs are allocated from an arena, and this is using xmalloc, so it's not zeroed. Labels have a position field that is not set in the new_label_body() function, so it could be zeroed or not depending on many things of course. When compiling the defined after a known compile-time false value, its labels are NOT added to the anchor, and so its position is not set during iseq_set_sequence in iseq_setup, but it is saved to the iseq's catch_table_ary. Then, during iseq_set_exception_table, the iseq_catch_table_entry's start and end are set to the LABEL's position because the LABEL is inside the iseq's catch_table_ary. There is no check for garbage values, which would be negative in this case, as position is an int. The iseq_catch_table_entry takes this possibly garbage value and saves it as its start and end.

I've updated my PR and added some assertions to the code to make sure this doesn't happen elsewhere.

These issues don't appear on current ruby master because prism is the new parser/compiler and it only affects the parse.y parser/compiler.

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed

[nobu (Nobuyoshi Nakada)]

• Backport changed from 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN to 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED

I'm not sure but this seems to need to be backported.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 3.1: REQUIRED, 3.2: REQUIRED, 3.3: REQUIRED to 3.1: WONTFIX, 3.2: WONTFIX, 3.3: REQUIRED

While trying to backport d592ddd5e619ffe1691b8050de2ccc3e1bd6e080 to ruby_3_2, I found that it depends on 6e64d4370456190541705ec4c6cf3af6bf4ac647 (for [Bug #19862]). And I cannot reproduce the SEGV on rub-3.2.
I decided to set WONTFIX for 3.1/3.2.
Please tell us if you found it can be reproduced on 3.2.

[k0kubun (Takashi Kokubun)]

Backporting this patch (https://github.com/ruby/ruby/pull/11554) to ruby_3_3 causes a lot of conflicts. Please file a PR to ruby_3_3 branch on GitHub for this ticket to be backported.