Bug #14096: Psych allows invalid single quote escape character

• Status: Closed
• Assignee: hsbt (Hiroshi SHIBATA)
• Target version: -
• ruby -v: 2.4.1
• Backport: 2.3: UNKNOWN, 2.4: UNKNOWN
• Tags: ext

[headius (Charles Nutter)]

In https://github.com/jruby/jruby/issues/4847 (and previously in https://github.com/jruby/jruby/issues/2199) we have had users report that escaped single quotes within a YAML string do not parse in JRuby, although they parse in MRI.

According to the YAML specs (both 1.1 and 1.2), \\' is not a valid escape character. The YAML library we use, SnakeYAML, has pushed back on adding it since it is not part of the spec.

Why does MRI parse this invalid YAML?

---
  !!seq [
    !!str "https://www.youtube.com/watch?v=DzpKasJJtRs",
    !!str "2Pac - Dont Care What Ya\'ll Think Remix Music Video 2017",
  ]

[headius (Charles Nutter)]

Link to https://github.com/yaml/libyaml/issues/68

[headius (Charles Nutter)]

Pull request https://github.com/yaml/libyaml/pull/74 has been created for libyaml, which means this is likely to percolate up to MRI over the coming years.

It might be good to get ahead of it and patch Psych to forbid the single quote escape, but I'm not sure how easy that would be to do.

We are closing related bugs in JRuby as "Won't Fix" given that this character is not spec and will soon be forbidden by libyaml.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned
• Assignee set to hsbt (Hiroshi SHIBATA)

[jeremyevans0 (Jeremy Evans)]

• Status changed from Assigned to Closed

This has been fixed in libyaml 0.2.1 (released June 2018), and the YAML given by @headius (Charles Nutter) will raise Psych::SyntaxError on Ruby versions 1.9-3.0 when used with libyaml 0.2.1+.