Project

General

Profile

Actions

Bug #6266

closed

encoding related exception with recent integrated psych

Added by jonforums (Jon Forums) over 10 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Target version:
ruby -v:
ruby 1.9.3p176 (2012-04-04 revision 35238) [i386-mingw32]
Backport:
[ruby-core:44163]

Description

Newer versions of Psych integrated into both ruby_1_9_3 and trunk create exceptions if the yaml file isn't perceived to be correctly encoded.

While I discovered this in my code that uses the 'logging' gem on Win7 32bit, this test snippet shows the same failure on both ruby_1_9_3@r35238 and trunk@r35229:

hello.yml uses LF-only line endings

C:\temp>type hello.yml
:greeting: hello yaml

yaml_test.rb uses LF-only

C:\temp>type yaml_test.rb
require 'yaml'
puts "YAML engine: #{YAML::ENGINE.yamler} #{'v'+Psych::VERSION if defined?(Psych)}"

fix: 'rb'

fix: 'r:utf-8'

fix: Encoding.default_external = Encoding::UTF_8 if defined?(Encoding)

fix: chcp 65001 in cmd.exe before running script

fix: run script with ruby -Eutf-8

File.open('hello.yml', 'r') do |f|
puts YAML.load_stream(f)
end

IBM437 codepage is a typical default for the cmd.exe shell

C:\temp>chcp
Active code page: 437

C:\temp>ruby -v yaml_test.rb
ruby 1.9.3p176 (2012-04-04 revision 35238) [i386-mingw32]
YAML engine: psych v1.3.1
C:/ruby193/lib/ruby/1.9.1/psych.rb:206:in parse': YAML file must be UTF-8, UTF-16LE, or UTF-16BE, not IBM437 (ArgumentError) from C:/ruby193/lib/ruby/1.9.1/psych.rb:206:in parse_stream'
from C:/ruby193/lib/ruby/1.9.1/psych.rb:289:in load_stream' from yaml_test.rb:10:in block in '
from yaml_test.rb:9:in open' from yaml_test.rb:9:in '

older versions of integrated psych do not cause exceptions

C:\temp>chcp
Active code page: 437

C:\temp>ruby -v yaml_test.rb
tcs-ruby 1.9.3p134 (2012-02-19, TCS patched 2012-02-25) [i386-mingw32]
YAML engine: psych v1.2.2
{:greeting=>"hello yaml"}

I've not tried on Arch with bash.

From the test_yaml.rb comments you can see some of the workarounds. None of these are good options because they require changes to either a users code/yaml, gem code that uses the psych library, or the ruby invocation command line.

For both trunk and ruby_1_9_3, I'd prefer psych implement a fallback strategy that Just Works combined with an informative warning rather than a hard exception.


Files

noname (500 Bytes) noname Anonymous, 04/07/2012 10:29 AM
Actions

Also available in: Atom PDF