Bug #20918: Prism error indicates line number of `-e` that does not exist

• Status: Closed
• Assignee: prism
• Target version: -
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[mame (Yusuke Endoh)]

$ ruby -e 'foo('
-e: -e:2: syntax error found (SyntaxError)
  1 | foo(
> 2 |
    | ^ unexpected end-of-input; expected a `)` to close the arguments

It says -e:2, but there is no line 2 in -e 'foo('.

eval("foo(") reports line 1, so I guess this issue is only for -e.

[ydah (Yudai Takada)]

It seems that the foo { has the same problem.

❯ ruby -e 'foo {'
-e: -e:2: syntax errors found (SyntaxError)
> 1 | foo {
    | ^ expected a block beginning with `{` to end with `}`
> 2 |
    | ^ unexpected end-of-input, assuming it is closing the parent top level context

Since the error is not confirmed until EOL, it looks like you are outputting the position where the error is confirmed.

❯ ruby -e "foo(

"
-e: -e:4: syntax error found (SyntaxError)
  2 |
  3 |
> 4 |
    | ^ unexpected end-of-input; expected a `)` to close the arguments

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned

[kddnewton (Kevin Newton)]

I looked into this this morning, it looks like ruby.c is automatically concatenating a \n onto the -e script here: https://github.com/ruby/ruby/blob/f43585b02c3634ab9a4e54049b08e04ab1a640fd/ruby.c#L1303. Is this desired behavior?

[tenderlovemaking (Aaron Patterson)]

kddnewton (Kevin Newton) wrote in #note-3:

I looked into this this morning, it looks like ruby.c is automatically concatenating a \n onto the -e script here: https://github.com/ruby/ruby/blob/f43585b02c3634ab9a4e54049b08e04ab1a640fd/ruby.c#L1303. Is this desired behavior?

I'm not sure if it matters. If you have a script with one line (that has a newline), Prism will report line 2 as the error.

[aaron@tc-lan-adapter ~/g/ruby (master)]$ cat x.rb
foo(
[aaron@tc-lan-adapter ~/g/ruby (master)]$ od -c x.rb
0000000    f   o   o   (  \n                                            
0000005
[aaron@tc-lan-adapter ~/g/ruby (master)]$ wc -l x.rb
       1 x.rb
[aaron@tc-lan-adapter ~/g/ruby (master)]$ ./miniruby x.rb
x.rb: x.rb:2: syntax error found (SyntaxError)
  1 | foo(
> 2 | 
    | ^ unexpected end-of-input; expected a `)` to close the arguments

[aaron@tc-lan-adapter ~/g/ruby (master)]$ 

It's fine for newlines to appear in method parameters as well as blocks, so Prism considers the EOF token to be the "error token". Indeed EOF "occurs" on line 2 (since it's after the newline), but since nobody writes EOF in to their files, I think we should consider the error to have occurred at the newline preceding EOF.

One more example:

[aaron@tc-lan-adapter ~/g/ruby (master)]$ cat x.rb
foo(




[aaron@tc-lan-adapter ~/g/ruby (master)]$ wc -l x.rb
       5 x.rb
[aaron@tc-lan-adapter ~/g/ruby (master)]$ od -c x.rb
0000000    f   o   o   (  \n  \n  \n  \n  \n                            
0000011
[aaron@tc-lan-adapter ~/g/ruby (master)]$ ./miniruby x.rb
x.rb: x.rb:6: syntax error found (SyntaxError)
  4 | 
  5 | 
> 6 | 
    | ^ unexpected end-of-input; expected a `)` to close the arguments

[aaron@tc-lan-adapter ~/g/ruby (master)]$

Most tools agree the above file only has 5 lines in it, but Prism reports the error on line 6. IMO errors should only occur on lines that exist.

[tenderlovemaking (Aaron Patterson)]

• Status changed from Assigned to Closed

Applied in changeset git|6877c38866b4213f5aa476223d21a4f4b5364247.

---

[ruby/prism] Fix error messages for unterminated ( and {

If we hit an EOF token, and the character before the EOF is a newline,
we should make EOF token start at the previous newline. That way any
errors reported will occur on that line.

For example "foo(\n" should report an error on line 1 even though the
EOF technically occurs on line 2.

[Bug #20918]
https://bugs.ruby-lang.org/issues/20918

https://github.com/ruby/prism/commit/60bc43de8e

[mame (Yusuke Endoh)]

• Status changed from Closed to Assigned

@tenderlovemaking (Aaron Patterson) Thank you for fixing the issue, but it still shows line 2 in the code snippet.

$ ruby -e "foo("
-e: -e:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments
  2 |

I don't think 2 | should be printed.

kddnewton (Kevin Newton) wrote in #note-3:

I looked into this this morning, it looks like ruby.c is automatically concatenating a \n onto the -e script here: https://github.com/ruby/ruby/blob/f43585b02c3634ab9a4e54049b08e04ab1a640fd/ruby.c#L1303. Is this desired behavior?

According to @nobu (Nobuyoshi Nakada), the new line is intentionally added to separate expressions for multiple -e options.

$ ruby -e 'p 1' -e 'p 2'
1
2

$ ruby -e 'p 1 +' -e '2'
3

However, I don't think it should be printed in an error message and snippet.

[tenderlovemaking (Aaron Patterson)]

mame (Yusuke Endoh) wrote in #note-6:

@tenderlovemaking (Aaron Patterson) Thank you for fixing the issue, but it still shows line 2 in the code snippet.

$ ruby -e "foo("
-e: -e:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments
  2 |

I don't think 2 | should be printed.

Strange. Thank you, I'll try to figure it out!

[tenderlovemaking (Aaron Patterson)]

• Status changed from Assigned to Closed

Applied in changeset git|e11c86f43e045462f4c0e2eaa2ddb4fdb6927ea7.

---

Fix error messages so we don't output an extra line

Before this commit, when a file ended with a newline, the syntax error
message would show an extra line after the file.

For example, the error message would look like this:

[aaron@tc-lan-adapter ~/g/ruby (master)]$ echo "foo(" > test.rb
[aaron@tc-lan-adapter ~/g/ruby (master)]$ od -c test.rb
0000000    f   o   o   (  \n
0000005
[aaron@tc-lan-adapter ~/g/ruby (master)]$ wc -l test.rb
       1 test.rb
[aaron@tc-lan-adapter ~/g/ruby (master)]$ ./miniruby test.rb
test.rb: test.rb:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments
  2 |

This commit fixes the "end of line" book keeping when printing an error
so that there is no extra line output at the end of the file:

[aaron@tc-lan-adapter ~/g/ruby (fix-last-line-error)]$ echo "foo(" | ./miniruby
-: -:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments

[aaron@tc-lan-adapter ~/g/ruby (fix-last-line-error)]$ echo -n "foo(" | ./miniruby
-: -:1: syntax error found (SyntaxError)
> 1 | foo(
    |     ^ unexpected end-of-input; expected a `)` to close the arguments

Notice that in the above example, the error message only displays one
line regardless of whether or not the file ended with a newline.

[Bug #20918]
[ruby-core:120035]