Project

General

Profile

Bug #10285

StringIO with encodings broken due to #9769

Added by dbussink (Dirkjan Bussink) about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
2.1.3
[ruby-core:65240]

Description

It looks like the change in https://bugs.ruby-lang.org/issues/9769 resulted in a behavior change with how StringIO works with different encodings.

The following snippet is broken and now raises:

test_string_io_encoding.rb:8:in `write': incompatible character encodings: ASCII-8BIT and Windows-1252 (Encoding::CompatibilityError)
    from test_string_io_encoding.rb:8:in `<main>'
require 'stringio'

io = StringIO.new
io.set_encoding(Encoding::ASCII_8BIT)
io.write("quz \x83 mat".force_encoding(Encoding::ASCII_8BIT))

str = "foo \x97 bar".force_encoding(Encoding::WINDOWS_1252)
io.write(str)

p io.string

What is the intended behavior here? If I change the code to not set the encoding on the StringIO object, it does work somehow:

require 'stringio'

io = StringIO.new
io.write("quz \x83 mat".force_encoding(Encoding::ASCII_8BIT))

str = "foo \x97 bar".force_encoding(Encoding::WINDOWS_1252)
io.write(str)

p io.string

In this case it sees io.string as UTF-8 encoded, but this is invalid. It does allow the second StringIO#write here though.

Associated revisions

Revision 2a8989d7
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47716 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 47716
Added by nobu (Nobuyoshi Nakada) about 5 years ago

stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Revision 10a4543f
Added by nagachika (Tomoyuki Chikanaga) about 5 years ago

merge revision(s) r47716: [Backport #10285]

    * ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO
      should be writable any encoding strings, without conversion.
      [ruby-core:65240] [Bug #10285]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_1@47982 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 47982
Added by nagachika (Tomoyuki Chikanaga) about 5 years ago

merge revision(s) r47716: [Backport #10285]

* ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO
  should be writable any encoding strings, without conversion.
  [ruby-core:65240] [Bug #10285]

Revision 897b2177
Added by usa (Usaku NAKAMURA) about 5 years ago

merge revision(s) 47716: [Backport #10285]

    * ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO
      should be writable any encoding strings, without conversion.
      [ruby-core:65240] [Bug #10285]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_0_0@47988 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 47988
Added by usa (Usaku NAKAMURA) about 5 years ago

merge revision(s) 47716: [Backport #10285]

* ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO
  should be writable any encoding strings, without conversion.
  [ruby-core:65240] [Bug #10285]

History

Updated by dbussink (Dirkjan Bussink) about 5 years ago

As another data point, I also tried this with writing to a file which seems to work:

File.open("out.txt", :mode => "w", :encoding => Encoding::ASCII_8BIT) do |f|
  str1 = "quz \x83 mat".force_encoding(Encoding::ASCII_8BIT)
  f.write(str1)
  str2 = "foo \x97 bar".force_encoding(Encoding::WINDOWS_1252)
  f.write(str2)
end

p File.read("out.txt")

Updated by usa (Usaku NAKAMURA) about 5 years ago

  • Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 2.0.0: REQUIRED, 2.1: REQUIRED

Updated by nobu (Nobuyoshi Nakada) about 5 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Applied in changeset r47716.


stringio.c: ASCII-8BIT StringIO rejects no encodings

  • ext/stringio/stringio.c (strio_write): ASCII-8BIT StringIO should be writable any encoding strings, without conversion. [ruby-core:65240] [Bug #10285]

Updated by nagachika (Tomoyuki Chikanaga) about 5 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED to 2.0.0: REQUIRED, 2.1: DONE

Backported into ruby_2_1 at r47982.

Updated by usa (Usaku NAKAMURA) about 5 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: DONE to 2.0.0: DONE, 2.1: DONE

Backported into ruby_2_0_0 at r47988.

Also available in: Atom PDF