Bug #17530: irb handles << incorrectly with variable as identifier

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN

[danfranklin (Dan Franklin)]

If you attempt to start a here-document with an identifier that is also a variable, it is interpreted as an append, which makes sense.
That is,

s1 = 'testing'
s2 = 'this'
s2 <<s1
adding text here does not work
s1

Gets an expected error:

testcase.rb:4: syntax error, unexpected local variable or method, expecting '('
adding text here does not work

But irb doesn't know this, and gathers input as though it's a here-document:

irb(main):001:0' s1 = 'testing'
=> "testing"
irb(main):002:0' s2 = 'this'
=> "this"
irb(main):003:0" s2 <<s1
irb(main):004:0" adding text here does not work
irb(main):005:0" s1
Traceback (most recent call last):
3: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `<main>'
2: from /home/centos/.rubies/ruby-3.0.0/bin/irb:23:in `load'
1: from /home/centos/.rubies/ruby-3.0.0/lib/ruby/gems/3.0.0/gems/irb-1.3.0/exe/irb:11:in `<top (required)>'
SyntaxError ((irb):4: syntax error, unexpected local variable or method, expecting '(')
adding text here does not work
^~~~
irb(main):006:0> 

Notice the prompt changes on lines 3-5 as though line 3 started a double-quoted here document.

irb should realize that line 3 is appending to s2, not starting a here-document.

This bug is present in the latest irb:

ruby --version
ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]

irb --version
irb 1.3.0 (2020-12-25)

It's an odd edge case, but it confused me when I tried to understand the double meaning of <<, so it seemed worth noting.

[jeremyevans0 (Jeremy Evans)]

This issue comes from Ripper, and is due to the fact that in irb, ripper is being called line-by-line and not all at once. When ripper parses the s2 <<s1 line, it doesn't know that s2 is a local variable, so it treats <<s1 as a heredoc (on_heredoc_beg) instead of << and s1 as separate tokens.

There are a few ways to fix this:

1. Modify ripper to somehow take a list of local variables to consider already in scope, and use that.
2. Modify irb to prepend the local variables in scope when using Ripper to evaluate each line.
3. Modify irb to check on on_heredoc_beg for preceding on_sp and on_ident tokens, and if the on_ident token is a local variable in scope, with similar changes for on_heredoc_end.

I thought approach 2 was the simplest to implement, so I've submitted a pull request for it: https://github.com/ruby/irb/pull/242

FWIW, the behavior that switches from heredoc to append is not that heredoc identifier (s1 in the example) is a local variable, but that the receiver of << (s2 in the example) is a local variable. If s2 is a method, the code is treated as a heredoc:

s1 = 'testing'
def s2(*a); p('this' + a.inspect) end
s2 <<s1
adding text here does not work
s1
# "this[\"adding text here does not work\\n\"]"

If s1 is a method, the code is treated as an append:

def s1(*a); p('testing' + a.inspect) end
s2 = 'this'
s2 <<s1
adding text here does not work
s1
# -:4: syntax error, unexpected local variable or method, expecting '('
# adding text here does not work

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed