Project

General

Profile

Backport #8323

no conversion by "bom|utf-8"

Added by nobu (Nobuyoshi Nakada) over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
[ruby-core:54563]

Description

Mode spec in open and etc. accepts BOM-prefix UTF encoding, however if the external and internal encodings given there are same no conversion take place regardless the actual external encoding.
Since the encoding prefixed with "BOM" is not a real encoding, but just a fallback, the conversion should honor the detected encoding by BOM.


Files

Associated revisions

Revision 91fb5bc8
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@40462 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 40462
Added by nobu (Nobuyoshi Nakada) over 6 years ago

io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Revision 4d3df433
Added by nagachika (Tomoyuki Chikanaga) over 6 years ago

merge revision(s) 40462: [Backport #8323]

    * io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is
      not a real encoding name, just a fallback.  so the proper conversion
      should take place even if if the internal encoding is equal to the
      bom-prefixed name, unless actual encoding is equal to the internal
      encoding.  [ruby-core:54563] [Bug #8323]

    * io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM
      found.  [ruby-core:54569]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_0_0@40541 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 40541
Added by nagachika (Tomoyuki Chikanaga) over 6 years ago

merge revision(s) 40462: [Backport #8323]

* io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is
  not a real encoding name, just a fallback.  so the proper conversion
  should take place even if if the internal encoding is equal to the
  bom-prefixed name, unless actual encoding is equal to the internal
  encoding.  [ruby-core:54563] [Bug #8323]

* io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM
  found.  [ruby-core:54569]

History

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • Description updated (diff)

Sorry, miss-post to ruby-core.

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • File 0001-io.c-conversion-from-bom-encoding.patch added

A patch attached

Updated by naruse (Yui NARUSE) over 6 years ago

The patch doesn't work on following case:

% ./ruby -e'IO.write"p","a";open("p","r:BOM|utf-8:utf-8"){|f|p f.read.size}'
-e:1:in read': code converter not found (UTF-8 to UTF-8) (Encoding::ConverterNotFoundError)
from -e:1:in
block in '
from -e:1:in open'
from -e:1:in
'

#5

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • File deleted (0001-io.c-conversion-from-bom-encoding.patch)

Updated by naruse (Yui NARUSE) over 6 years ago

nobu (Nobuyoshi Nakada) wrote:

Updated.

OK, commit please.

#7

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r40462.
Nobuyoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


io.c: conversion from bom encoding

  • io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is not a real encoding name, just a fallback. so the proper conversion should take place even if if the internal encoding is equal to the bom-prefixed name, unless actual encoding is equal to the internal encoding. [ruby-core:54563] [Bug #8323]
  • io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM found. [ruby-core:54569]

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN to 1.9.3: REQUIRED, 2.0.0: REQUIRED
#9

Updated by nobu (Nobuyoshi Nakada) over 6 years ago

  • Tracker changed from Bug to Backport
  • Project changed from Ruby master to Backport200
  • Category deleted (M17N)
  • Status changed from Closed to Assigned
  • Assignee changed from naruse (Yui NARUSE) to nagachika (Tomoyuki Chikanaga)
  • Target version deleted (2.1.0)
#10

Updated by nagachika (Tomoyuki Chikanaga) over 6 years ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r40541.
Nobuyoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


merge revision(s) 40462: [Backport #8323]

* io.c (rb_io_ext_int_to_encs, parse_mode_enc): bom-prefixed name is
  not a real encoding name, just a fallback.  so the proper conversion
  should take place even if if the internal encoding is equal to the
  bom-prefixed name, unless actual encoding is equal to the internal
  encoding.  [ruby-core:54563] [Bug #8323]

* io.c (io_set_encoding_by_bom): reset extenal encoding if no BOM
  found.  [ruby-core:54569]

Also available in: Atom PDF