Bug #20270: Options with `--parser=prism`

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE

[nobu (Nobuyoshi Nakada)]

--dump option

Currently parsetree and prism_parsetree bits are separated, but it seems meaningless as far as --parser option selects only one parser.

Why doesn't simply --dump=parsetree dump AST by parse.y, or PRISM AST if --parser=prism is given?

Actually, ruby --dump=parsetree --parser=prism just segfaults, because these bits and --parser option are separately handled.

streaming code from stdin

$ echo end | ruby --parser=prism -
ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
-: warning: Prism support for streaming code from stdin is not currently supported

Nothing reported is fine, since it is not currently supported.

I was a bit curious that it looks like trying something after the warning.

            if (strcmp(opt->script, "-") == 0) {
                int xflag = opt->xflag;
                VALUE rb_source = open_load_file(opt->script_name, &xflag);
                opt->xflag = xflag != 0;

                rb_warn("Prism support for streaming code from stdin is not currently supported");
                error = pm_parse_string(&result, rb_source, opt->script_name);
            }

Note that open_load_file returns rb_stdin when - is given.
This object is a RFile, not a RString, and RSTRING_PTR(rb_source) accesses after the object boundary.
It should just bail out, not try obviously wrong thing.

Fix

https://github.com/nobu/ruby/tree/options-refactor
https://github.com/ruby/ruby/pull/9991

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)

JFYI: It just so happens that the second issue is not a segmentation fault.

In pm_parse_string call just after the warning:

(lldb) rp source
bits: [    U]
(struct RFile) $1 = {
  basic = (flags = 11, klass = 4301593320)
  fptr = 0x0000000128628c10
}
(lldb) p *(struct RString*)source
(struct RString) {
  basic = (flags = 11, klass = 4301593320)
  len = 4972514320
  as = {
    heap = {
      ptr = 0x0000000000000000
      aux = (capa = 0, shared = 0)
    }
    embed = (ary = "")
  }
}

source is actually a RFile, but is interpreted as a huge embedded RString that starts with a NUL char.
Its as.embed is beyond the boundary since VWA, it is unpredictable what can be there.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

[k0kubun (Takashi Kokubun)]

The diff contains refactoring changes and the cherry-pick of the associated changes doesn't apply cleanly to ruby_3_3. Could you file a backport PR to ruby_3_3 branch?

[k0kubun (Takashi Kokubun)]

• Status changed from Closed to Open

[k0kubun (Takashi Kokubun)]

• Status changed from Open to Closed

[k0kubun (Takashi Kokubun)]

• Backport changed from 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED to 3.0: DONTNEED, 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE

ruby_3_3 97b1bf9ac11848c2783264d22bf7cdb7f32a21cf.