Bug #1311

YAML::dump creates strings YAML::load cannot parse

Added by rampion (Noah Easterly) about 3 years ago. Updated about 1 year ago.

[ruby-core:23006]
Status:Closed Start date:03/24/2009
Priority:Normal Due date:
Assignee:- % Done:

100%

Category:-
Target version:-
ruby -v:ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.9.0], ruby 1.8.7 (2008-05-31 patchlevel 0) [i686-darwin9.3.0]

Description

This case seems to occur whenever
* the string begins with a newline, and 
* the last line doesn't end in whitespace (except, perhaps, newlines), and
* the second line (after the initial whitespace) begins with more whitespace than some later line

Here are a couple example strings:
* "\n a\nb"
* "\n a\n  b\nc"
* "\n  a\n b"

YAML::load throws an ArgumentError with a syntax error on the line that begins with less leading
whitespace.  
* YAML.load YAML.dump("\n a\nb") #=> ArgumentError: syntax error on line 3, col 0: `b'
* YAML.load YAML.dump("\n a\n  b\nc") #=> ArgumentError: syntax error on line 4, col 0: `c'
* YAML.load YAML.dump("\n  a\n b") #=> ArgumentError: syntax error on line 3, col 1: ` b'

This has been observed in ruby-1.8.6p0 and ruby-1.8.7p0.

Here is another observation of this same bug: http://blog.smartlogicsolutions.com/2008/09/04/ruby-patch-to-fix-broken-yamldump-for-multi-line-strings-stringto_yaml/

syck_scan_scalar.patch (497 Bytes) daz (Dave B), 03/25/2009 09:54 pm

test_yaml.patch - all tests pass (1.3 kB) daz (Dave B), 03/25/2009 09:54 pm

test_yaml2.patch (1.4 kB) coatl (caleb clausen), 09/30/2009 02:23 am

syck_scan_scalar2.patch (540 Bytes) coatl (caleb clausen), 09/30/2009 02:23 am

yaml-error.txt (1.4 kB) weppos (Simone Carletti), 03/19/2010 09:02 pm


Related issues

duplicates ruby-trunk - Bug #2767: YAML bug Closed 02/20/2010

Associated revisions

Revision 27591
Added by nobu (Nobuyoshi Nakada) about 2 years ago

* ext/syck/emitter.c (syck_scan_scalar): set SCAN_WHITEEDGE flag when scalar begins with newline. patches from Dave B <daz AT d10.karoo.co.uk> at [ruby-core:23019] and caleb clausen at [ruby-core:25851]. [ruby-core:23006][ruby-core:29925]

History

Updated by rampion (Noah Easterly) about 3 years ago

Correction: should read "the second line (after the initial *newline*)"

Updated by daz (Dave B) about 3 years ago

Good reports, thanks.

I've found a workaround but I'm not sure enough about the YAML spec to guarantee that it's the correct fix.

The patch is mild - it just forces one style over another in the same way as the existing code and allows the new tests to pass along with the rest.

Two patches to 1.9.2dev - these files don't change often ;)

?CHANGELOG: Wed Mar 25 20:06:10 2009  Dave B <daz@d10.karoo.co.uk>
           * ext/syck/emitter.c:  Set SCAN_WHITEEDGE flag when
             scalar begins with newline [ruby-core:23006][Bug #1311]

           * test/yaml/test_yaml.rb: add new tests and comment old
             Test::Unit::TestSuite code to enable them to run as $0

Updated by coatl (caleb clausen) over 2 years ago

I have also been bitten by this same bug. In addition to the cases described above, this problem also occurs when the input contains just a single newline (in either unix or dos style). The patch supplied by Dave B fixes the case of a string containing just a single unix newline, but fails to fix a string with just a single dos newline (and probably the other cases initially reported, if the unix newlines are converted to dos newlines). I'm attaching slightly a slightly modified patches, which addresses the dos newline issue, and tests single dos and unix newlines.

This problem also occurs in ruby 1.9.

Updated by daz (Dave B) over 2 years ago

Caleb's revisions are the way to go.

( test_yaml2.patch & syck_scan_scalar2.patch )

Harmlessly beneficial - no compatibility issues - emits YAML to the spec.


daz

Updated by weppos (Simone Carletti) about 2 years ago

Here's an other test case for a similar issue.

require 'yaml'

> c = File.read("yaml-error.txt")
> YAML.load(YAML.dump(c))
'rgumentError: syntax error on line 34, col 0: `-- 
	from /Users/weppos/.rvm/ruby-1.8.7-p248/lib/ruby/1.8/yaml.rb:133:in `load'
	from /Users/weppos/.rvm/ruby-1.8.7-p248/lib/ruby/1.8/yaml.rb:133:in `load'
	from (irb):3

Updated by mame (Yusuke Endoh) about 2 years ago

  • Status changed from Open to Closed
Hi,

Syck is deprecated, in effect.  This issue won't be fixed.
Instead, try Psych.  [ruby-core:29337]

-- 
Yusuke Endoh <mame@tsg.ne.jp>

Updated by coatl (caleb clausen) about 2 years ago

Yusuke Endoh and I have verified that all of the test cases reported here are fixed when using psych. For proof of most, see [ruby-core:29323]. And I just now verified the last test case is now ok, submitted by Simone Carletti. (No console dump, sorry.)

Updated by nobu (Nobuyoshi Nakada) about 2 years ago

  • Status changed from Closed to Open

Updated by nobu (Nobuyoshi Nakada) about 2 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100
This issue was solved with changeset r27591.
Noah, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

Also available in: Atom PDF