Project

General

Profile

Actions

Misc #17662

open

The heredoc pattern used in tests does not syntax highlight correctly in many editors

Added by Eregon (Benoit Daloze) about 3 years ago. Updated over 2 years ago.


Description

This heredoc pattern

    assert_ruby_status([], "#{<<-"begin;"}\n#{<<-'end;'}", bug)
    begin;
      exit("1" == Thread.start(1, &:to_s).value)
    end;

completely breaks syntax highlighting in at least:

  • GitHub: there are many examples
  • Atom
  • RubyMine (and IntelliJ)
  • Likely many more editors based on TextMate grammars

Could another pattern be used in tests inside the ruby/ruby repository (at least for test/ruby)?

Due to this issue, it is very annoying and inconvenient to look at/read/investigate many tests.

I think this pattern is also very complicated to understand (and using ; is quite weird for this).
I suggest to replace it with this obvious and simple pattern many people use:

    assert_ruby_status([], <<~'RUBY', bug)
      exit("1" == Thread.start(1, &:to_s).value)
    RUBY

This syntax highlights correctly in most (all?) editors, and as an added bonus the code inside the heredoc is also highlighted in some editors (due to the label being RUBY).

Actions #1

Updated by Eregon (Benoit Daloze) about 3 years ago

  • Subject changed from The herdoc pattern used in tests does not syntax highlight correctly in many editors to The heredoc pattern used in tests does not syntax highlight correctly in many editors

Updated by xtkoba (Tee KOBAYASHI) about 3 years ago

In the suggested pattern, the here document seems not highlighted nor auto-indented by Emacs Ruby Mode. Would you please also suggest the configuration of Emacs Ruby Mode for the suggested pattern?

Updated by Eregon (Benoit Daloze) about 3 years ago

I'm not an Emacs user, so I don't know.
I think indentation doesn't matter much, it could be this too:

    assert_ruby_status([], <<~'RUBY', bug)
    exit("1" == Thread.start(1, &:to_s).value)
    RUBY

Highlighting the Ruby code inside the heredoc seems a nice extra, but the issue of not highlighting the entire file due to the pattern seems much worse.

I guess the best would be if the Emacs Ruby Mode implements highlighting if the heredoc label is RUBY.
I'm open to see other patterns that might work better on Emacs and still don't destroy syntax highlighting of the file in other editors.

Updated by xtkoba (Tee KOBAYASHI) about 3 years ago

Emacs Ruby Mode behaves well with the current pattern. I wonder why the other editors do not simulate it.

Updated by marcandre (Marc-Andre Lafortune) about 3 years ago

It's a really complex formulation: a double HEREDOC inside string interpolatiosn, with a delimiter of the HEREDOC using double quotes but clearly that's not the end of the string because it's inside a #{}...
I can barely makes sense of it myself.

Updated by mame (Yusuke Endoh) about 3 years ago

  • Status changed from Open to Assigned
  • Assignee set to nobu (Nobuyoshi Nakada)

My vim works great with the code :-) But breaking syntax highligt in GitHub is not so great.

AFAIK the style is mainly written by @nobu (Nobuyoshi Nakada). Nobu, what do you think?

Updated by nobu (Nobuyoshi Nakada) about 3 years ago

I don't mind to transfer to <<~RUBY if ruby-mode.el supporting it gets released.
BTW, what editors support it?

Updated by nobu (Nobuyoshi Nakada) about 3 years ago

Also, does auto indentation work inside <<~RUBY as begin;...end; hack?

Updated by mame (Yusuke Endoh) about 3 years ago

Okay, the style is an intentional hack to trick the editors into syntax-highlinting and auto-indenting the code within the heredoc. I didn't know, but it makes sense to me.

I investigated the syntax highlighting behavior of GitHub. They does not correctly support an empty heredoc in an indent:

https://gist.github.com/mame/90c1427163266ba73fab4cf7d96edd0a

I think they should fix it, but I found one workaround: adding a new line in <<"begin;" heredoc solves the issue.

https://github.com/ruby/ruby/blob/d10b535806227d24a3f8f5c7f425be3a39c2ac74/test/ruby/test_string.rb#L697

Updated by mame (Yusuke Endoh) about 3 years ago

mame (Yusuke Endoh) wrote in #note-9:

I investigated the syntax highlighting behavior of GitHub. They does not correctly support an empty heredoc in an indent:

https://gist.github.com/mame/90c1427163266ba73fab4cf7d96edd0a

Wow, it looks like that GitHub fixed the issue. The code is now colored correctly as a heredoc. Great.

Eregon (Benoit Daloze) wrote:

completely breaks syntax highlighting in at least:

These lines are also syntax-highlighted correctly.

For the record: when this ticket was created, all the code after the heredoc was not colored at all in GitHub file view.

Updated by marcandre (Marc-Andre Lafortune) about 3 years ago

Great for Github!

FWIW, both VS Code and Sublime Text get very confused by this pattern.

Updated by Eregon (Benoit Daloze) over 2 years ago

Here is a particularly weird/complex example usage of heredocs in tests:
https://github.com/ruby/ruby/blob/eef3c08edc2fc74496cdc1e4d89798d053e32561/test/ruby/test_thread.rb#L1380-L1394

It does not syntax highlight, it is very hard to read, and the {#1 nesting seems there for no reason after 2 lines of code?
The inner script is actually just:

code = <<~RUBY
tpid = #{Process.pid}
sig = :#{sig}
STDOUT.sync = true
while gets
  puts
  Process.kill(sig, tpid)
end
RUBY

I think it would be great to consolidate to a single style of heredoc in tests,
and if possible one that does not involve twice the number of heredocs than needed.

Updated by nobu (Nobuyoshi Nakada) over 2 years ago

Eregon (Benoit Daloze) wrote in #note-12:

and if possible one that does not involve twice the number of heredocs than needed.

It is intentional to indent inside here-docs and separate interpolation-enabled and -disabled parts.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0