Project

General

Profile

Backport #2762

in `sub!': invalid byte sequence in UTF-8 (ArgumentError); using backticks

Added by JonnyJD (Johannes Dewender) about 9 years ago. Updated almost 8 years ago.

Status:
Assigned
Priority:
Normal
[ruby-core:28246]

Description

=begin
Current stalbe ruby 1.9.1 has problems with UTF/Unicode strings coming from backticks.
Reading the file with File.read('filename') works as expected. Only using backticks gives an error in a substitution afterwards:
in `sub!': invalid byte sequence in UTF-8 (ArgumentError)

This is my test skript:
#!/usr/bin/env ruby
puts "Using ruby #{RUBY_VERSION} on #{RUBY_PLATFORM} from #{RUBY_RELEASE_DATE}"
puts "File.read:"
File.read('testString.txt').sub!(/foo./, "hallo")
puts " success"
puts "backticks:"
cat testString.txt.sub!(/foo.
/, "hallo")
puts " success"
puts "No problem in this version"

And this is the output on the ruby versions I tested:

Using ruby 1.8.7 on x86_64-linux from 2010-01-10
File.read:
success
backticks:
success
No problem in this version

Using ruby 1.9.1 on x86_64-linux from 2010-01-10
File.read:
success
backticks:
./test.rb:9:in sub!': invalid byte sequence in UTF-8 (ArgumentError)
from ./test.rb:9:in

Using ruby 1.9.2 on x86_64-linux from 2010-02-13
File.read:
success
backticks:
success
No problem in this version

(using revision 26652 for 1.9.2)

So it is working in 1.9.2 again, but could this fix get backported to stable 1.9.1? I could not find out which revision this is fixed though, because I had trouble compiling older revisions.
=end


Files

testString.txt (43 KB) testString.txt test file giving an error with test.rb on 1.9.1 JonnyJD (Johannes Dewender), 02/19/2010 07:13 AM
test.rb (302 Bytes) test.rb test script JonnyJD (Johannes Dewender), 02/19/2010 07:13 AM

History

#1

Updated by JonnyJD (Johannes Dewender) about 9 years ago

=begin
The test script is a minimal case of course. The problem actually showed up together with ruby-gettext (which is using the backticks) and a program named rubyripper:
http://code.google.com/p/rubyripper/issues/detail?id=400

The test file is a russian po file for gettext translation. I could not find UTF problems myself in this file and isutf8 from some moreutils package can't find a problem either.
=end

#2

Updated by naruse (Yui NARUSE) almost 9 years ago

  • Status changed from Open to Assigned
  • Assignee set to yugui (Yuki Sonoda)

=begin
This is r25880. [ruby-dev:39743]
=end

Also available in: Atom PDF