Bug #4383: psych fails to parse a symbol in a flow sequence

• Status: Third Party's Issue
• Assignee: tenderlovemaking (Aaron Patterson)
• Target version: 1.9.3
• ruby -v: ruby 1.9.3dev (2011-02-08 trunk 30821) [i386-darwin9.8.0]

[yugui (Yuki Sonoda)]

=begin
% ruby-trunk -rsyck -ryaml -ve 'p YAML.load("order: [ :year, :month, :day ]")'
ruby 1.9.3dev (2011-02-08 trunk 30821) [i386-darwin9.8.0]
{"order"=>[:year, :month, :day]}

% ruby-trunk -rpsych -ryaml -ve 'p YAML.load("order: [ :year, :month, :day ]")'
ruby 1.9.3dev (2011-02-08 trunk 30821) [i386-darwin9.8.0]
/Users/yugui/local/lib/ruby-trunk/1.9.1/psych.rb:153:in parse': (<unknown>): couldn't parse YAML at line 0 column 10 (Psych::SyntaxError) from /Users/yugui/local/lib/ruby-trunk/1.9.1/psych.rb:153:in parse_stream'
from /Users/yugui/local/lib/ruby-trunk/1.9.1/psych.rb:124:in parse' from /Users/yugui/local/lib/ruby-trunk/1.9.1/psych.rb:111:in load'
from -e:1:in `'

The example YAML is a part of lib/active_support/locale/en.yml in ActiveSupport 2.3.10.
=end

[godfat (Lin Jen-Shin)]

=begin
I wonder if this is not a valid YAML, should we allow it?
According to http://yaml-online-parser.appspot.com/

ERROR:
while parsing a flow node
expected the node content, but found ':'
in "", line 1, column 10:
order: [ :year, :month, :day ]

This works though:

order:

• :year
• :month
• :day
  =end

[yugui (Yuki Sonoda)]

=begin
According to YAML 1.1 spec. http://yaml.org/spec/1.1, any flow node can occur as an element of a flow sequence, a ns-flow-node(n,c) can be a ns-plain-single(c) where c = flow-in, and a ns-plain-single(c) can start with ':' by definition of ns-plain-first-char(c).

I think it is a valid YAML 1.1. Is it wrong?
=end

[tenderlovemaking (Aaron Patterson)]

=begin
On Tue, Feb 08, 2011 at 10:07:09PM +0900, Yuki Sonoda wrote:

Issue #4383 has been updated by Yuki Sonoda.

According to YAML 1.1 spec. http://yaml.org/spec/1.1, any flow node can occur as an element of a flow sequence, a ns-flow-node(n,c) can be a ns-plain-single(c) where c = flow-in, and a ns-plain-single(c) can start with ':' by definition of ns-plain-first-char(c).

I think it is a valid YAML 1.1. Is it wrong?

I think you are correct. However, that means this is a bug in libyaml.
I've emailed the libyaml mailing list to find out if we're interpreting
the spec correctly.

--
Aaron Patterson
http://tenderlovemaking.com/

Attachment: (unnamed)
=end

[headius (Charles Nutter)]

=begin
JRuby, using a different library SnakeYAML (itself a port of libyaml) exhibits the same error:

~/projects/jruby ➔ jruby --1.9 -rpsych -ryaml -ve 'p YAML.load("order: [ :year, :month, :day ]")'
jruby 1.6.0.RC2 (ruby 1.9.2 patchlevel 136) (2011-02-08 3d4720a) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_22) [darwin-x86_64-java]
org/jruby/ext/psych/PsychParser.java:242:in parse': while parsing a flow node; expected the node content, but found Value (RuntimeError) from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:148:in parse_stream'
from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:119:in parse' from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:106:in load'
from -e:1:in `(root)'
=end

[tenderlovemaking (Aaron Patterson)]

=begin
On Thu, Feb 10, 2011 at 04:21:05AM +0900, Charles Nutter wrote:

Issue #4383 has been updated by Charles Nutter.

JRuby, using a different library SnakeYAML (itself a port of libyaml) exhibits the same error:

~/projects/jruby ➔ jruby --1.9 -rpsych -ryaml -ve 'p YAML.load("order: [ :year, :month, :day ]")'
jruby 1.6.0.RC2 (ruby 1.9.2 patchlevel 136) (2011-02-08 3d4720a) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_22) [darwin-x86_64-java]
org/jruby/ext/psych/PsychParser.java:242:in parse': while parsing a flow node; expected the node content, but found Value (RuntimeError) from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:148:in parse_stream'
from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:119:in parse' from /Users/headius/projects/jruby/lib/ruby/1.9/psych.rb:106:in load'
from -e:1:in `(root)'

Yay for consistency? ;-)

Looks like this is a bug in both libyaml and SnakeYAML. According to
other people on the yaml-core list, this example should be valid:

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTimfwVXs_qbWV-Tqng_6HYwVSU_FSkQGWT2KLE9n%40mail.gmail.com&forum_name=yaml-core
http://yaml.org/ypaste/802/index.html

It seems that colons in flow sequences cause ambiguity problems. The
python community has already encountered these problems (yay for sharing
code) and have started a wiki page to try coming to a solution:

http://pyyaml.org/wiki/YAMLColonInFlowContext

I will add this example to the wiki.

As this is a bug in libyaml (and SnakeYAML), should we close this ticket
as a third party issue?

--
Aaron Patterson
http://tenderlovemaking.com/

Attachment: (unnamed)
=end

[tenderlovemaking (Aaron Patterson)]

• Status changed from Assigned to Third Party's Issue

=begin
I'm closing this as a third party issue. The problem stems from ambiguities in the YAML spec with regard to JSON and YAML support. This exception was added on purpose in libyaml. Please read the discussion on the libyaml mailing list for more information:

http://sourceforge.net/mailarchive/forum.php?thread_name=AANLkTi%3DtRBNOo__zmn%3D%3D3hW_VhQQNG1WeD5E-1z6ZAKm%40mail.gmail.com&forum_name=yaml-core

In the mean time, I am updating projects with YAML that isn't compliant.
=end