Bug #11988: YAML.dump doesn't quote string starting with 0 which will be recognized as float in YAML 1.2

• Status: Closed
• Assignee: tenderlovemaking (Aaron Patterson)
• Target version: -
• ruby -v: ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin14]
• Backport: 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED, 2.3: DONE

[frsyuki (Sadayuki Furuhashi)]

YAML.dump doesn't quote string "019" but it is recognized as float using Core Schema tag resolution option of YAML 1.2 specification.
Therefore, SnakeYAML (Java) recognizes a string "019" serialized by Ruby's YAML as float 19.0 unexpectedly.

YAML.dump works as following:

irb(main):002:0> require 'yaml'
=> true
irb(main):003:0> puts YAML.dump({'a' => '017'})
---
a: '017'
=> nil
irb(main):004:0> puts YAML.dump({'a' => '019'})
---
a: 019
=> nil

This is valid behavior as a YAML 1.1 implementation.
However, YAML 1.2 defines a new optional mechanism for resolving types implicitly. With this new option named Core Schema, type of 019 is resolved as tag:yaml.org,2002:float.
Here is the specification: http://www.yaml.org/spec/1.2/spec.html#id2805071

irb(main):002:0> /[-+]?(\.[0-9]+|[0-9]+(\.[0-9]*)?)([eE][-+]?[0-9]+)?/ =~ '019'
=> 0

I think it's better to quote 019 as '019' to avoid unexpected results when YAML is read by YAML 1.2 Core Schema implementation.

Related: a rejected pull-request to SnakeYAML
https://bitbucket.org/asomov/snakeyaml/pull-requests/4/fix-implicit-resolver-of-float-type-so/diff

[nobu (Nobuyoshi Nakada)]

I suspect that it may be a behavior of libyaml.

[naruse (Yui NARUSE)]

• Status changed from Open to Assigned
• Assignee set to tenderlovemaking (Aaron Patterson)
• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED, 2.3: REQUIRED

I found it is psych's issue.
Following fixes this (but may be it is fixed in Psych::ScalarScanner#tokenize.

diff --git a/ext/psych/lib/psych/visitors/yaml_tree.rb b/ext/psych/lib/psych/visitors/yaml_tree.rb
index 7c40050..15cd89c 100644
--- a/ext/psych/lib/psych/visitors/yaml_tree.rb
+++ b/ext/psych/lib/psych/visitors/yaml_tree.rb
@@ -331,7 +331,8 @@ def visit_String o
           style = Nodes::Scalar::FOLDED
         elsif o =~ /^[^[:word:]][^"]*$/
           style = Nodes::Scalar::DOUBLE_QUOTED
-        elsif not String === @ss.tokenize(o)
+        elsif not String === @ss.tokenize(o) or /\A0[0-7]*[89]/ =~ o
+          p @ss
           style = Nodes::Scalar::SINGLE_QUOTED
         end

[hsbt (Hiroshi SHIBATA)]

I addressed this issue to upstream. ref https://github.com/tenderlove/psych/pull/270

[hsbt (Hiroshi SHIBATA)]

• Status changed from Assigned to Closed

Applied in changeset r55497.

---

• ext/psych/, test/psych/: Upate psych 2.1.0
  This version fixed [Bug #11988][ruby-core:72850]

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED, 2.3: REQUIRED to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED, 2.3: DONE

ruby_2_3 r55902 merged revision(s) 55497,55498,55504.