Bug #13662: prohibit machine stackoverflow while GC

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN

[ko1 (Koichi Sasada)]

We have encounter GC marking miss after [Feature #13637] r59047 at test-rubyspec.

I, [2017-06-15T04:31:33.467582 #2820] INFO -- : ........................................................................................................................................................................................................................................................................................................................................................................................................../home/ko1/ruby/src/trunk/spec/rubyspec/core/encoding/converter/convpath_spec.rb:7: [BUG] rb_gc_mark(): 0x0000000001cbbf78 is T_NONE

http://ci.rvm.jp/results/trunk@P895/1497485282
http://ci.rvm.jp/results/trunk@P895/1497479452
http://ci.rvm.jp/results/trunk@P895/1497468696
... and more (not frequent).

My current assumption is machine stack overflow while GC and finishing marking phase incomplete.
To prevent such situation (incomplete GC), I'll prohibit machine stack overflow while running GC.

Index: vm_insnhelper.c
===================================================================
--- vm_insnhelper.c	(revision 59088)
+++ vm_insnhelper.c	(working copy)
@@ -56,8 +56,11 @@
 void
 rb_threadptr_stack_overflow(rb_thread_t *th)
 {
+    if (rb_during_gc()) {
+	rb_fatal("machine stack overflow while GC is running.");
+    }
 #ifdef USE_SIGALTSTACK
-    threadptr_stack_overflow(th, !rb_threadptr_during_gc(th));
+    threadptr_stack_overflow(th, TRUE);
 #else
     threadptr_stack_overflow(th, FALSE);
 #endif

[ko1 (Koichi Sasada)]

• Status changed from Open to Closed

Applied in changeset trunk|r59089.

---

Prohibit SystemStackError while GC.

• vm_insnhelper.c (rb_threadptr_stack_overflow): fatal on GC [Bug #13662]