Backport #7255

Ripper inside class_eval ommits :string_content between new line and :string_embexpr

Added by Micha J over 1 year ago. Updated over 1 year ago.

[ruby-core:48703]
Status:Closed
Priority:Normal
Assignee:Usaku NAKAMURA

Description

See the following Code parsed by Ripper using

content = 'class_eval <<-RUBY, __FILE__, __LINE__+1        

    def #{parent}contents(*args)
      ""
    end
RUBY
'

Ripper::SexpBuilder.new(content).parse

The resulting sexp ommits truncates the first string_add to " \n" instead of " \n def "

        - - :string_literal
          - - :string_add
            - - :string_add
              - - :string_add
                - - :string_content
                - - :@tstring_content
                  - "        \n"
                  - - 2
                    - 0

If you remove the empty line, it works fine and the " def " is part of the string. As well it works without any embed #{}.

Associated revisions

Revision 38319
Added by Usaku NAKAMURA over 1 year ago

merge revision(s) 38235: [Backport #7255]

* parse.y (parser_here_document): flush string content between new
  line and :string_embexpr.   [Bug #7255]

History

#1 Updated by Micha J over 1 year ago

I noticed, that this bug reaches a bit wider:

'class_eval <<-RUBY, __FILE__, __LINE__+1        

    def #{parent}_contents(*args)
      test = ""
      test ||= "#{parent}" 
    end
RUBY
'

' test ||= "' also is omitted by ripper. This is the string between the "\n" of [:@tstringcontent, "contents(*args)\n test = \"\"\n", [3, 21]]] and the next embed.

#2 Updated by Micha J over 1 year ago

Another update: It's not connected to class_eval but to the <<-RUBY and <<RUBY multiline String versions. This is my only use case for this format so I didn't notice.

#3 Updated by Micha J over 1 year ago

I did some further digging regarding the parser/scanner events.

Basically the cause for the error can be seen here (parsing the example from the description).

[:@ident, "class_eval", [1, 0]]
[:@sp, " ", [1, 10]]
[:@heredoc_beg, "<<-RUBY", [1, 11]]
[:string_content]
[:@tstring_content, "        \n", [2, 0]]
[:string_add, [:string_content], [:@tstring_content, "        \n", [2, 0]]]
[:@embexpr_beg, "        def \#{", [3, 0]]   <------- this line
...

The scanner-event :@embexprbeg contains the missing text which then will be discarded to create the stringembexp from the sexp.

#4 Updated by Zachary Scott over 1 year ago

  • Category set to ext
  • Assignee set to Nobuyoshi Nakada

#5 Updated by Nobuyoshi Nakada over 1 year ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r38235.
Micha, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


parse.y: flush string content

  • parse.y (parserheredocument): flush string content between new line and :string_embexpr. [Bug #7255]

#6 Updated by Nobuyoshi Nakada over 1 year ago

  • Tracker changed from Bug to Backport
  • Project changed from ruby-trunk to Backport93
  • Category deleted (ext)
  • Status changed from Closed to Assigned
  • Assignee changed from Nobuyoshi Nakada to Usaku NAKAMURA

#7 Updated by Usaku NAKAMURA over 1 year ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r38319.
Micha, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 38235: [Backport #7255]

* parse.y (parser_here_document): flush string content between new
  line and :string_embexpr.   [Bug #7255]

Also available in: Atom PDF