Backport #2762

in `sub!': invalid byte sequence in UTF-8 (ArgumentError); using backticks

Added by Johannes Dewender almost 2 years ago. Updated 10 months ago.

[ruby-core:28246]
Status:Assigned Start date:02/19/2010
Priority:Normal Due date:
Assignee:Yuki Sonoda % Done:

0%
Category:-
Target version:-

Description

Current stalbe ruby 1.9.1 has problems with UTF/Unicode strings coming from backticks.
Reading the file with File.read('filename') works as expected. Only using backticks gives an error in a substitution afterwards:
in `sub!': invalid byte sequence in UTF-8 (ArgumentError)

This is my test skript:
#!/usr/bin/env ruby
puts "Using ruby #{RUBY_VERSION} on #{RUBY_PLATFORM} from #{RUBY_RELEASE_DATE}"
puts "File.read:"
File.read('testString.txt').sub!(/foo.*/, "hallo")
puts "  success"
puts "backticks:"
`cat testString.txt`.sub!(/foo.*/, "hallo")
puts "  success"
puts "No problem in this version"


And this is the output on the ruby versions I tested:

Using ruby 1.8.7 on x86_64-linux from 2010-01-10
File.read:
  success
backticks:
  success
No problem in this version

Using ruby 1.9.1 on x86_64-linux from 2010-01-10
File.read:
  success
backticks:
./test.rb:9:in `sub!': invalid byte sequence in UTF-8 (ArgumentError)
        from ./test.rb:9:in `<main>

Using ruby 1.9.2 on x86_64-linux from 2010-02-13
File.read:
  success
backticks:
  success
No problem in this version

(using revision 26652 for 1.9.2)


So it is working in 1.9.2 again, but could this fix get backported to stable 1.9.1? I could not find out which revision this is fixed though, because I had trouble compiling older revisions.

testString.txt - test file giving an error with test.rb on 1.9.1 (43 kB) Johannes Dewender, 02/19/2010 07:13 am

test.rb - test script (302 Bytes) Johannes Dewender, 02/19/2010 07:13 am

History

#1
Updated by Johannes Dewender almost 2 years ago

The test script is a minimal case of course. The problem actually showed up together with ruby-gettext (which is using the backticks) and a program named rubyripper:
http://code.google.com/p/rubyripper/issues/detail?id=400

The test file is a russian po file for gettext translation. I could not find UTF problems myself in this file and isutf8 from some moreutils package can't find a problem either.

#2
Updated by Yui NARUSE almost 2 years ago

  • Status changed from Open to Assigned
  • Assignee set to Yuki Sonoda
This is r25880. [ruby-dev:39743]

Also available in: Atom PDF