Bug #8445

IO.open and IO#set_enconding does not support :fallback option

Added by Haruhiro Yoshimoto 11 months ago. Updated 3 months ago.

[ruby-dev:47369]
Status:Assigned
Priority:Normal
Assignee:Akira Tanaka
Category:M17N
Target version:current: 2.2.0
ruby -v:trunk(ruby 2.1.0dev) Backport:1.9.3: UNKNOWN, 2.0.0: UNKNOWN

Description

RubyDoc says that IO.open and IO#set_encoding supports optional argument defined in String#encode.
http://ruby-doc.org/core-2.0/IO.html#method-c-new-label-Options
In fact, :invalid, :undef and :replace works as expected.

However, :fallback option does not work neither for IO.open and IO#set_encoding.
Following is the example code which does not work.
f(x) is never called even if hoge.txt contains non convertible character.

File.open("./hoge.txt","r:ShiftJIS:utf-8", :fallback => lambda{|x|f(x)}){|f|
...
}
File.open("./hoge.txt"){|f|
f.set
encoding("Shift_JIS","utf-8",:fallback => lambda{|x|f(x)})
...
}

I Think this is because fillcbuf() in io.c calls rbeconvconvert() from transcode.c directly.
On the other hand, fallback
func is called in transcodeloop(), which is called by strencode().

Since transcodeloop() also calls rbeconvconvert(), I wrote a small patch which moves some codes from
transcode
loop() to rbeconvconvert() to fix the problem.

The attached file is the patch. Hope this helps.

support-fallback-for-io.patch Magnifier (3.9 KB) Haruhiro Yoshimoto, 05/24/2013 10:03 PM

History

#1 Updated by Yui NARUSE 11 months ago

  • Category set to M17N
  • Status changed from Open to Assigned
  • Assignee set to Akira Tanaka
  • Target version set to 2.1.0

Is this intentional or just unexpected inconsistency?

#2 Updated by Hiroshi SHIBATA 3 months ago

  • Target version changed from 2.1.0 to current: 2.2.0

Also available in: Atom PDF