Bug #12939: MRI crashes when a hash table becomes huge

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN

[vmakarov (Vladimir Makarov)]

Checking huge hash tables on the trunk, I found a bug in my code.
Here is the test case:

h = {}; 0x100000000.times{|i| h[i] = i}

MRI diagnostics for the test case is at the report end.

You need a machine with huge memory (at least 64 GB on my
estimation) to reproduce the bug. I used a machine with 256 GB memory.

So there is no sense to add a test (make test) for the bug.

The probability for the bug occurrence is quite small. Still it would
be nice to fix it. The patch in the attachment fixes the problem.

I also have some further memory and performance improvements for hash
tables but I am going to submit it later for 2.5 release.

---

ruby 2.4.0dev (2016-11-09 trunk 56677) [x86_64-linux]


-e:1: [BUG] Segmentation fault at 0x00000000000000
ruby 2.4.0dev (2016-11-09 trunk 56677) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0004 p:0013 s:0017 e:000016 BLOCK  -e:1 [FINISH]
c:0003 p:---- s:0012 e:000011 CFUNC  :times
c:0002 p:0015 s:0008 E:001990 EVAL   -e:1 [FINISH]
c:0001 p:0000 s:0003 E:001b90 (none) [FINISH]

-- Ruby level backtrace information ----------------------------------------
-e:1:in `<main>'
-e:1:in `times'
-e:1:in `block in <main>'

-- Machine register context ------------------------------------------------
 RIP: 0x00007f845d2d7a98 RBP: 0x00007f845e2691a0 RSP: 0x00007ffc4d65c2e0
 RAX: 0x0000000000000000 RBX: 0x00007f845e26b350 RCX: 0xffffffff7fffffff
 RDX: 0x548703a059b0d177 RDI: 0x0000000000000002 RSI: 0x548703a0d9b0d177
  R8: 0x00007f7c5589f028  R9: 0x0000000000000001 R10: 0x00007f7a5589e010
 R11: 0x00007f745589e010 R12: 0x0000000020000000 R13: 0x0000000000000001
 R14: 0x00007f7a5589e010 R15: 0x0000000000000002 EFL: 0x0000000000010246

-- C level backtrace information -------------------------------------------
/home/vmakarov/GITRUBY/build-tor/miniruby(rb_vm_bugreport+0x4f3) [0x7f845d362c53] /home/cygnus/vmakarov/GITRUBY/trunk/vm_dump.c:679
/home/vmakarov/GITRUBY/build-tor/miniruby(rb_bug_context+0xd4) [0x7f845d1d1084] /home/cygnus/vmakarov/GITRUBY/trunk/error.c:426
/home/vmakarov/GITRUBY/build-tor/miniruby(sigsegv+0x3e) [0x7f845d2ce56e] /home/cygnus/vmakarov/GITRUBY/trunk/signal.c:897
/lib64/libpthread.so.0 [0x7f845cd060d0]
/home/vmakarov/GITRUBY/build-tor/miniruby(rebuild_table+0x248) [0x7f845d2d7a98] /home/cygnus/vmakarov/GITRUBY/trunk/st.c:433
/home/vmakarov/GITRUBY/build-tor/miniruby(st_update+0x4bd) [0x7f845d2d9f0d] /home/cygnus/vmakarov/GITRUBY/trunk/st.c:1098
/home/vmakarov/GITRUBY/build-tor/miniruby(rb_hash_aset+0xcf) [0x7f845d1fa38f] /home/cygnus/vmakarov/GITRUBY/trunk/hash.c:574
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_exec_core+0x4984) [0x7f845d34e4e4] /home/cygnus/vmakarov/GITRUBY/trunk/insns.def:1857
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_exec+0x8b) [0x7f845d35039b] /home/cygnus/vmakarov/GITRUBY/trunk/vm.c:1712
/home/vmakarov/GITRUBY/build-tor/miniruby(rb_yield_1+0x40a) [0x7f845d359f7a] /home/cygnus/vmakarov/GITRUBY/trunk/vm.c:969
/home/vmakarov/GITRUBY/build-tor/miniruby(int_dotimes+0x3e) [0x7f845d23767e] /home/cygnus/vmakarov/GITRUBY/trunk/numeric.c:4916
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_call_cfunc+0x105) [0x7f845d348315] /home/cygnus/vmakarov/GITRUBY/trunk/vm_insnhelper.c:1752
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_call_method+0xe3) [0x7f845d352463] /home/cygnus/vmakarov/GITRUBY/trunk/vm_insnhelper.c:2288
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_exec_core+0x1195) [0x7f845d34acf5] /home/cygnus/vmakarov/GITRUBY/trunk/insns.def:967
/home/vmakarov/GITRUBY/build-tor/miniruby(vm_exec+0x8b) [0x7f845d35039b] /home/cygnus/vmakarov/GITRUBY/trunk/vm.c:1712
/home/vmakarov/GITRUBY/build-tor/miniruby(ruby_exec_internal+0xb2) [0x7f845d1d6eb2] /home/cygnus/vmakarov/GITRUBY/trunk/eval.c:244
/home/vmakarov/GITRUBY/build-tor/miniruby(ruby_run_node+0x2f) [0x7f845d1da69f] /home/cygnus/vmakarov/GITRUBY/trunk/eval.c:308
/home/vmakarov/GITRUBY/build-tor/miniruby(main+0x4b) [0x7f845d15addb] /home/cygnus/vmakarov/GITRUBY/trunk/eval_intern.h:173
...

• Status changed from Open to Closed

Applied in changeset r56793.

---

st.c: fix crashes on huge hash tables

From: Vladimir Makarov [e-mail]

By Vladimir's estimation, this requires at least 64 GB of memory
to reproduce this bug due to the hash sizes required. So there
is no new test case (and I am unable to test it, myself).

• st.c (get_bins_num): avoid out-of-bounds on shift by using correct type
  [ruby-core:78139] [Bug #12939]
• st.c (get_allocated_entries): ditto

[normalperson (Eric Wong)]

Thanks, committed as r56793