Bug #13736: ruby -00 should be the same as setting $/=""

• Status: Closed
• Assignee: matz (Yukihiro Matsumoto)
• Target version: -
• ruby -v: 2.4.1p111
• Backport: 2.2: REQUIRED, 2.3: DONE, 2.4: DONE

[dawg (Andrew Dumke)]

Suppose you have blocks of text separated by 2 or more \n. A typical text file with records defined by a black line.

Given:

$ cat lines
f1, r1
f2, r1 then 2 \n:

f1, r2 then 3 \n:


f1,r3
f2,r3 then 4 \n:



f1, r4
f2,r4 then 6 \n: 





f1,r5

The script $ ruby -00 -F"\n" -lane 'END{p $.}' lines SHOULD have the same number of records as $ ruby -F"\n" -lane 'BEGIN{$/=""}; END{p $.}' lines. It does not not.

The script $ ruby -00 -F"\n" -lane 'END{p $.}' lines SHOULD have the same number of record as $ perl -00 -F"\n" -lane 'END{print $.}' lines Again, it does not.

The script $ ruby -00 -F"\n" -lane 'END{p $.}' lines shows 8. The other scripts here show 5 -- the correct number.

The behavior of the -00 command switch is not the same as perl's -00 command switch. It is also not the same as setting $/="" in either ruby or perl.

The \n\n pattern between blocks should be a single record separator even if you have \n\n\n\n\n That is true in perl, awk, gawk, and ruby with $/="". It is not true with ruby -00. This is a bug.

[dawg (Andrew Dumke)]

Please see Stackoverflow https://stackoverflow.com/q/44999907/298607 for a better / more complete description.

[dawg (Andrew Dumke)]

• File deleted (lines)

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)
• Status changed from Open to Assigned
• Assignee set to matz (Yukihiro Matsumoto)

I agree that it's a bug because ruby.1 states -00 turns Ruby into paragraph mode.
I'm uncertain the reason though, this behavior has been there since 0.95 at least, but 0.49 behaved same as perl.

[nobu (Nobuyoshi Nakada)]

diff --git i/ruby.c w/ruby.c
index b2b2c690ff..4497331d48 100644
--- i/ruby.c
+++ w/ruby.c
@@ -1162,7 +1162,7 @@ proc_options(long argc, char **argv, ruby_cmdline_options_t *opt, int envopt)
 		if (v > 0377)
 		    rb_rs = Qnil;
 		else if (v == 0 && numlen >= 2) {
-		    rb_rs = rb_str_new2("\n\n");
+		    rb_rs = rb_str_new2("");
 		}
 		else {
 		    c = v & 0xff;
diff --git i/test/ruby/test_rubyoptions.rb w/test/ruby/test_rubyoptions.rb
index 93eecd28c4..a2e2a99e19 100644
--- i/test/ruby/test_rubyoptions.rb
+++ w/test/ruby/test_rubyoptions.rb
@@ -187,6 +187,8 @@
     assert_in_out_err(%w(-0e) + ["print gets"], "foo\nbar\0baz", %W(foo bar\0), [])
 
     assert_in_out_err(%w(-00 -e) + ["p gets, gets"], "foo\nbar\n\nbaz\nzot\n\n\n", %w("foo\nbar\n\n" "baz\nzot\n\n"), [])
+
+    assert_in_out_err(%w(-00 -e) + ["p gets, gets"], "foo\nbar\n\n\n\nbaz\n", %w("foo\nbar\n\n" "baz\n"), [])
   end
 
   def test_autosplit

[nobu (Nobuyoshi Nakada)]

• Status changed from Assigned to Closed

Applied in changeset trunk|r59970.

---

ruby.c: paragraph mode by -00

• ruby.c (proc_options): set to paragraph mode, if -00 is given,
  as well as perl and -R0 option in 0.49.
  [ruby-core:81987] [Bug #13736]

[dawg (Andrew Dumke)]

This does not seem to be corrected on ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin16]

Exactly the same results as before.

[dawg (Andrew Dumke)]

• Status changed from Closed to Feedback

Try:

  $ printf "1\n1b\n\n\n\n\n2\n2b\n\n3\3b" | ruby -00 -F"\n" -lane 'END{p $.}'
 4

vs

 $ printf "1\n1b\n\n\n\n\n2\n2b\n\n3\3b" | ruby -F"\n" -lane 'BEGIN{$/=""}; END{p $.}'
 3

vs

 $ printf "1\n1b\n\n\n\n\n2\n2b\n\n3\3b" | perl -00 -F"\n" -lane 'END{print $.}'
 3

Try with the original text from lines.txt

Sorry -- not fixed...

[nobu (Nobuyoshi Nakada)]

• Status changed from Feedback to Closed
• Backport changed from 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN to 2.2: REQUIRED, 2.3: REQUIRED, 2.4: REQUIRED

As I asked matz about the intention and then committed, this has not been backported yet.

WARNING: the condition that "Status = Closed" and "Backport is REQUIRED" is the trigger to backport, it will never get backported if you change the status.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.2: REQUIRED, 2.3: REQUIRED, 2.4: REQUIRED to 2.2: REQUIRED, 2.3: REQUIRED, 2.4: DONE

ruby_2_4 r62666 merged revision(s) 59970.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.2: REQUIRED, 2.3: REQUIRED, 2.4: DONE to 2.2: REQUIRED, 2.3: DONE, 2.4: DONE

ruby_2_3 r62817 merged revision(s) 59970.