Bug #18903: Stack overflow signal handling seems to be triggered once and then not working after

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [arm64-darwin21]
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

[chrisseaton (Chris Seaton)]

This program creates a recursive object graph and then tries to convert it to JSON with no max depth, so it stack overflows in C code that does no co-operative stack overflow checks, as the bytecode interpreter would. This therefore triggers a segmentation fault and the stack overflow detection there. It works the first time, but the second time around it doesn't and the program hard crashes on M1.

Is there something like a guard page permission that is switched during the handling, and needs to switched back for the guard page to work again?

Note that it isn't JSON specific - I think any stack overflow within C code would do it.

require 'json'

a = []
a << a

begin
  JSON.dump(a)
rescue Exception
  puts 'rescued'
end

JSON.dump(a)

[chrisseaton (Chris Seaton)]

(Found by Jean Boussier)

[Eregon (Benoit Daloze)]

In general it is not possible to recover from a stack overflow (be it in C or Ruby), the executing program should be considered hopelessly corrupted because e.g. it might have happened in the middle of a critical section.
So IMHO the right fix would be to make stack overflows not rescue-able and hard exit on such a case.
That's from my experience with stack overflows on the JVM, some details are different on CRuby but the overall problem is the same.

[mame (Yusuke Endoh)]

Discussed at the dev meeting. @nobu (Nobuyoshi Nakada) said he would investigate if he could fix it with M2.

Recovering from a stack overflow in C is not portable and complete, but it is a fact that they work almost well in Linux, which is the main production environment. If we make it a fatal error that cannot be rescued normally, it might impact on users of Rails apps, job queues like Sidekiq, etc.

[mame (Yusuke Endoh)]

• Has duplicate Bug #21373: "Illegal instruction: 4" when raising SystemStackError after rescuing added

[soulcutter (Bradley Schaefer)]

One observation I have is that there are ways to prevent this in application code:

require 'json'
circular_reference_hash = {}.tap { |h| h[:self] = h }

# Globally
JSON.dump_default_options[:max_nesting] = 100
JSON.dump(circular_reference_hash)

# More-specifically
#   JSON.dump(obj, io = nil, limit = nil)
JSON.dump(circular_reference_hash, nil, 100)

I picked 100, because that happens to be the default max_nesting option in JSON.generate (and also to_json, which uses the same defaults). I guess I'm wondering why dump has unsafe defaults in comparison to generate - maybe dump's defaults could match generate's (at least for max_nesting)?