Project

General

Profile

Bug #13350

File.read :newline option not respected on Linux

Added by larskanis (Lars Kanis) about 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
-
ruby -v:
ruby 2.5.0dev (2017-03-21 trunk 58044) [x86_64-linux]
[ruby-core:80270]

Description

Reproducable in irb

All examples run with:

Encoding.default_external = 'utf-8'
Encoding.default_internal = nil

on Linux (unexpected behavior)

Prepare a file with CRLF newline characters

File.write("crlf.txt", "a\nb\nc", newline: :crlf) # => 7
File.binread("crlf.txt") # => "a\r\nb\r\nc"

While read the files newline characters are only converted if internal and external encoding differ like here:

File.read("crlf.txt", mode: "r:utf-8:cp850", newline: :universal) # => "a\nb\nc"

But if the encoding isn't changed, the newline character is not converted:

File.read("crlf.txt", newline: :universal) # => "a\r\nb\r\nc"
File.read("crlf.txt", mode: "r:utf-8:utf-8", newline: :universal) # => "a\r\nb\r\nc"
File.open("crlf.txt", "r:utf-8:utf-8", newline: :universal, &:read) # => "a\r\nb\r\nc"

I would expect to get:

File.read("crlf.txt", newline: :universal) # => "a\nb\nc"

... but the :newline option seems to be ignored, when read without encoding conversion.

on Windows (expected behavior)

In contrast, on Windows the same commands returns in only 5 bytes to be written, but the content of the file is the same as on Linux:

File.write("crlf.txt", "a\nb\nc", newline: :crlf) # => 5
File.binread("crlf.txt") # => "a\r\nb\r\nc"

Due to the default CRLF->LF conversion on Windows, the newline option isn't necessary to get a CRLF->LF conversion. But setting newline to :cr shows, that the :newline option is respected as expected.

File.read("crlf.txt", newline: :universal) # => "a\nb\nc"
File.read("crlf.txt", newline: :cr) # => "a\r\nb\r\nc"

Associated revisions

Revision 57464618
Added by nobu (Nobuyoshi Nakada) almost 2 years ago

io.c: textmode if newline decorator

  • io.c (validate_enc_binmode): newline decorator implies text mode now. [ruby-core:80270] [Bug #13350]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59336 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 59336
Added by nobu (Nobuyoshi Nakada) almost 2 years ago

io.c: textmode if newline decorator

  • io.c (validate_enc_binmode): newline decorator implies text mode now. [ruby-core:80270] [Bug #13350]

Revision 59336
Added by nobu (Nobuyoshi Nakada) almost 2 years ago

io.c: textmode if newline decorator

  • io.c (validate_enc_binmode): newline decorator implies text mode now. [ruby-core:80270] [Bug #13350]

Revision 59336
Added by nobu (Nobuyoshi Nakada) almost 2 years ago

io.c: textmode if newline decorator

  • io.c (validate_enc_binmode): newline decorator implies text mode now. [ruby-core:80270] [Bug #13350]

History

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

newline: universal does not imply text mode.
...but it seems nice even if the newline: option enables it.

Updated by matz (Yukihiro Matsumoto) almost 2 years ago

It seems to be a nice idea to turn text-mode on with newline option.

Matz.

Updated by hsbt (Hiroshi SHIBATA) almost 2 years ago

  • Assignee set to nobu (Nobuyoshi Nakada)
  • Status changed from Open to Assigned
#4

Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

  • Status changed from Assigned to Closed

Applied in changeset trunk|r59336.


io.c: textmode if newline decorator

  • io.c (validate_enc_binmode): newline decorator implies text mode now. [ruby-core:80270] [Bug #13350]

Also available in: Atom PDF