Bug #19547: `RubyVM::AST.parse("\\", keep_tokens: true)` dumps core

• Status: Closed
• Assignee: yui-knk (Kaneko Yuichiro)
• Target version: -
• ruby -v: ruby 3.3.0dev (2023-03-25T08:17:05Z master 9bc2dbd33c) [x86_64-linux]
• Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[mame (Yusuke Endoh)]

$ ./miniruby -e 'RubyVM::AbstractSyntaxTree.parse("\\", keep_tokens: true)'
<internal:ast>:59: [BUG] parser_token2id: unknown token 92
ruby 3.3.0dev (2023-03-25T08:17:05Z master 9bc2dbd33c) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0003 p:0003 s:0015 e:000014 METHOD <internal:ast>:59
c:0002 p:0008 s:0006 e:000005 EVAL   -e:1 [FINISH]
c:0001 p:0000 s:0003 E:001db0 DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
-e:1:in `<main>'
<internal:ast>:59:in `parse'

-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- C level backtrace information -------------------------------------------
/home/mame/work/ruby/miniruby(rb_print_backtrace+0xd) [0x55ee562be9be] /home/mame/work/ruby/vm_dump.c:785
/home/mame/work/ruby/miniruby(rb_vm_bugreport) /home/mame/work/ruby/vm_dump.c:1101
/home/mame/work/ruby/miniruby(bug_report_end+0x0) [0x55ee560ab94f] /home/mame/work/ruby/error.c:790
/home/mame/work/ruby/miniruby(rb_bug_without_die) /home/mame/work/ruby/error.c:790
/home/mame/work/ruby/miniruby(die+0x0) [0x55ee560092a1] /home/mame/work/ruby/error.c:798
/home/mame/work/ruby/miniruby(rb_bug) /home/mame/work/ruby/error.c:800
/home/mame/work/ruby/miniruby(rb_long2int_inline+0x0) [0x55ee5600a925] /home/mame/work/ruby/parse.y:630
/home/mame/work/ruby/miniruby(add_delayed_token) /home/mame/work/ruby/parse.y:7008
/home/mame/work/ruby/miniruby(add_delayed_token) /home/mame/work/ruby/parse.y:6997
/home/mame/work/ruby/miniruby(parser_dispatch_scan_event+0x25) [0x55ee56174855] /home/mame/work/ruby/parse.y:6222
/home/mame/work/ruby/miniruby(parser_dispatch_scan_event) /home/mame/work/ruby/parse.y:6212
/home/mame/work/ruby/miniruby(yylex+0xf) [0x55ee56184173] /home/mame/work/ruby/parse.y:10492
/home/mame/work/ruby/miniruby(ruby_yyparse) /home/mame/work/ruby/parse.c:9357
/home/mame/work/ruby/miniruby(yycompile0+0xf9) [0x55ee561954c9] /home/mame/work/ruby/parse.y:6767
/home/mame/work/ruby/miniruby(rb_suppress_tracing+0x10c) [0x55ee562c369c] /home/mame/work/ruby/vm_trace.c:483
/home/mame/work/ruby/miniruby(yycompile+0x55) [0x55ee5617b76c] /home/mame/work/ruby/parse.y:6824
/home/mame/work/ruby/miniruby(parser_compile_string) /home/mame/work/ruby/parse.y:6892
/home/mame/work/ruby/miniruby(rb_parser_compile_string_path) /home/mame/work/ruby/parse.y:6905
/home/mame/work/ruby/miniruby(ast_parse_done+0x0) [0x55ee560279a8] /home/mame/work/ruby/ast.c:103
/home/mame/work/ruby/miniruby(rb_ast_parse_str) /home/mame/work/ruby/ast.c:104
/home/mame/work/ruby/miniruby(ast_s_parse) /home/mame/work/ruby/ast.c:90
/home/mame/work/ruby/miniruby(invoke_bf+0x12) [0x55ee562ae7fe] /home/mame/work/ruby/vm_insnhelper.c:6414
/home/mame/work/ruby/miniruby(vm_invoke_builtin_delegate) /home/mame/work/ruby/vm_insnhelper.c:6441
/home/mame/work/ruby/miniruby(vm_exec_core) /home/mame/work/ruby/insns.def:1526
/home/mame/work/ruby/miniruby(rb_vm_exec+0x938) [0x55ee562a14e8] /home/mame/work/ruby/vm.c:2366
/home/mame/work/ruby/miniruby(rb_ec_exec_node+0xb1) [0x55ee560b0ff1] /home/mame/work/ruby/eval.c:287
/home/mame/work/ruby/miniruby(ruby_run_node+0x4c) [0x55ee560b634c] /home/mame/work/ruby/eval.c:328
/home/mame/work/ruby/miniruby(rb_main+0x21) [0x55ee560126c6] ./main.c:39
/home/mame/work/ruby/miniruby(main) ./main.c:58
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_call_main+0x80) [0x7fdc30c23510] ../sysdeps/nptl/libc_start_call_main.h:58
/lib/x86_64-linux-gnu/libc.so.6(call_init+0x0) [0x7fdc30c235c9] ../csu/libc-start.c:381
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main_impl) ../csu/libc-start.c:368
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main) (null):0
[0x55ee56012715]

[nobu (Nobuyoshi Nakada)]

https://github.com/nobu/ruby/tree/backslash-token
https://github.com/ruby/ruby/pull/7622

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed

Applied in changeset git|6f122965cf8704f019445faead58040e9be2effb.

---

[Bug #19547] Add token for unescaped backslash

This token is exposed only when RubyVM::AbstractSyntaxTree with
keep_tokens option.