Bug #20518: Escaped-newline in %W

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN

[akr (Akira Tanaka)]

I found an escaped-newline in %W literal is interpreted as a newline character.

% ./ruby -e '
p %W[a\
b]'
["a\nb"]

I expected it to be interpreted as a line continuation but actually not.

I'm considering to describe \<newline> as a kind of escape sequence in the document.
(Related to https://bugs.ruby-lang.org/issues/20503#note-4 )
Unfortunately, this behavior contradicts: \<newline> doesn't behave as usual escape sequences.

Note that %W is interpolable string-array literal.
https://docs.ruby-lang.org/en/master/syntax/literals_rdoc.html#label-25w+and+-25W-3A+String-Array+Lite

"interpolable" means that interpolation and escape sequences are enabled.

Other escape sequences and interpolation work well.

% ./ruby -e '
p %W[a\
b c\x21d e#{6*7}f]'
["a\nb", "c!d", "e42f"]

\x21 is interpreted as "!" and #{6*7} is iterpreted as "42".
It is the usual interpretation of escape sequences and interpolation.
But \<newline> is interpreted as "\n".
It is not the usual interpretation of escape sequences (line continuation).

It is interpreted as a line continuation in a double-quoted string:

% ./ruby -e '
p %Q[a\
b c\x21d e#{6*7}f]'
"ab c!d e42f"

\<newline> is interpreted as empty (line continuation) as expected.
\x21 is interpreted as "!" and #{6*7} is interpreted as "42".

I found this behavior changed between Ruby 1.8.0 and 1.8.1.
Ruby 1.8.0 interprets \<newline> as line continuation.

% ruby-1.8.0 -e '
p %W[a\
b c\x21d e#{6*7}f]'
["ab", "c!d", "e42f"]
% ruby-1.8.1 -e '
p %W[a\
b c\x21d e#{6*7}f]'
["a\nb", "c!d", "e42f"]

I think this is the commit to change the behavior.

commit e168b64b03403fb4efbd8dcbbdaeed83710b4d39 0d1896a88fe091000ce0d9c8ef3cf7ef1e2e991b
Author: Nobuyoshi Nakada <[e-mail]>
Date:   2003-09-04 14:59:43 +0000

    * parse.y (tokadd_string): newlines have no special meanings in
      %w/%W, otherwise they are ignored only when interpolation is
      enabled.  [ruby-dev:21325]


    git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@4496 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

I read the discussion from [ruby-dev:21325].
https://public-inbox.org/ruby-dev/5FD2F0CF7F5D7F44B00F36870B9E78B508DE5071@SBG-EX4/

It discusses about %w.
%W is not considered well.

The behavior of %w is also changed between Ruby 1.8.0 and 1.8.1.

% ruby-1.8.0 -e '
p %w[a\
b c\x21d e#{6*7}f]'
["ab", "c\\x21d", "e#{6*7}f"]
% ruby-1.8.1 -e '
p %w[a\
b c\x21d e#{6*7}f]'
["a\nb", "c\\x21d", "e#{6*7}f"]

I understand the intent of this behavior for %w: it makes it possible to include white spaces in the word.
Without this behavior, there is no way to include a newline character in the word.

But I don't understand the intent for %W.
Including a newline character using \n is possible because escape sequences are enabled.

% ./ruby -v
ruby 3.4.0dev (2024-05-25T10:15:25Z master 0bae2f0002) [x86_64-linux]

[akr (Akira Tanaka)]

%I has the same issue.

% ruby -e '
p %I[a\
b c\x21d e#{6*7}f]'
[:"a\nb", :"c!d", :e42f]
% ruby -e '
p %i[a\
b c\x21d e#{6*7}f]'
[:"a\nb", :"c\\x21d", :"e\#{6*7}f"]

%I inteprets \<newline> as a newline, not line continuation.

[akr (Akira Tanaka)]

• Status changed from Open to Closed

Applied in changeset git|5e1001f754b34e1f0cc67563512c6036b6eb75ab.

---

[DOC] document line continuation.

Document details of escape sequences including line continuation.

[Bug #20518]