Project

General

Profile

Actions

Feature #2043

closed

incompatible character encodings

Feature #2043: incompatible character encodings

Added by vo.x (Vit Ondruch) about 16 years ago. Updated over 14 years ago.

Status:
Rejected
Target version:
[ruby-core:25360]

Description

=begin
Why the following example fails with the "Encoding::CompatibilityError: incompatible character encodings: Windows-1250 and UTF-8" exception?

s = "\u017Elu\u0165ou\u010dk\u00fd"
a = s.encode 'cp1250'
b = s.encode 'utf-8'
c = a + b

I would expect that if the strings are not in the same encoding, that Ruby will do everything they can to satisfy me, but they just tries if there is possible conversion to ASCII otherwise exception is fired. This is really annoying behavior.

Have you considered to allow such string merge?
=end

Updated by naruse (Yui NARUSE) about 16 years ago Actions #1

  • Status changed from Open to Assigned
  • Assignee set to naruse (Yui NARUSE)

=begin
Sorry, what is "possible conversion to ASCII" ?
=end

Updated by yugui (Yuki Sonoda) about 16 years ago Actions #2

  • Target version set to 3.0

=begin

=end

Updated by vo.x (Vit Ondruch) about 16 years ago Actions #3

=begin
In following example, just characters from US-ASCII are used and in this case the addition works well.

s = 'abc'
a = s.encode 'cp1250'
b = s.encode 'utf-8'
c = a + b
=end

Updated by naruse (Yui NARUSE) about 16 years ago Actions #4

=begin
Ruby 1.9 doesn't automatic conversion.
ASCII character set is a special
because those characters of ASCII compatible encodings are the same characters.

On Ruby 1.9's view, Unicode is not a superset of Windows-1252.
=end

Updated by nobu (Nobuyoshi Nakada) about 16 years ago Actions #5

  • Status changed from Assigned to Rejected

=begin

=end

Updated by vo.x (Vit Ondruch) about 16 years ago Actions #6

=begin

On Ruby 1.9's view, Unicode is not a superset of Windows-1252.

Is the "Ruby 1.9's view" somewhere described in detail? I still have the feeling that it is just half baked :/
=end

Updated by naruse (Yui NARUSE) about 16 years ago Actions #7

=begin
http://jp.rubyist.net/magazine/?0025-Ruby19_m17n
http://yokolet.blogspot.com/2009/07/design-and-implementation-of-ruby-m17n.html
I wrote above but in Japanese, second is its translation.

http://github.com/candlerb/string19/tree/master
James' and string19 is also well documented.

People in ISO 8859 may think why Unicode is not a super set of Windows-1252.
In Japan, because of lack of standard conversion tables
between Japanese legacy encoding (Shift_JIS, EUC-JP, ISO-2022-JP) and Unicode,
vendors use different tables.
This sad situation made that Unicode is not a simple super set of legacy.
Ruby 1.9 inherits this.

If wide consensus for the standard table was made before Ruby 2.0,
Ruby 2.0 may have automatic conversion (or Unicode comes to be the internal code).
=end

Updated by vo.x (Vit Ondruch) about 16 years ago Actions #8

=begin
Thank you for the links. It was interesting.

I'm looking forward Ruby 2.0 and their automatic conversions, since writing c = a.encode('utf-8') + b.encode('utf-8') to safely concatenate two strings is not sexy at all.

Vit
=end

Actions

Also available in: PDF Atom