Backport #6212: Backport bug fixes for psych

• Status: Closed
• Assignee: -

[tenderlovemaking (Aaron Patterson)]

I'd like to backport bugfixes I've made on psych, including fixing a memory leak, dealing with encodings, and parsing partial documents.

I would like to apply this patch to the 1.9.3 branch (and also increase the version number).

Can I get a review of this patch?

[bitsweat (Jeremy Daer)]

LOOKS GREAT!

I'm grateful to see these fixes backported to stdlib psych.

[drbrain (Eric Hodel)]

I have reviewed this patch and it looks good to me.

• Status changed from Open to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r35165.
Aaron, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

merge revision(s) 32578,33401,33403,33404,33531,33655,33679,33809,33900,33965,34067,34069,34087,34328,34330,34527,34772,34783,34839,34914,34953,34954,35153: [Backport #6212]

* ext/psych/lib/psych.rb: updating version to match gem
* ext/psych/psych.gemspec: ditto
* ext/psych/lib/psych/visitors/to_ruby.rb: fixing deprecation warning

* ext/psych/lib/psych.rb: define a new BadAlias error class.

* ext/psych/lib/psych/visitors/to_ruby.rb: raise an exception when
  deserializing an alias that does not exist.

* test/psych/test_merge_keys.rb: corresponding test.

* ext/psych/lib/psych.rb (load, parse): stop parsing or loading after
  the first document has been parsed.

* test/psych/test_stream.rb: pertinent tests.

* ext/psych/lib/psych.rb (parse_stream, load_stream): if a block is
  given, documents will be yielded to the block as they are parsed.
  [ruby-core:42404] [Bug #5978]

* ext/psych/lib/psych/handlers/document_stream.rb: add a handler that
  yields documents as they are parsed

* test/psych/test_stream.rb: corresponding tests.

* ext/psych/lib/psych/core_ext.rb: only extend Kernel if IRB is loaded
  in order to stop method pollution.

* ext/psych/lib/psych.rb: default open YAML files with utf8 external
  encoding. [ruby-core:42967]
* test/psych/test_tainted.rb: ditto

* ext/psych/parser.c: prevent a memory leak by protecting calls to
  handler callbacks.
* test/psych/test_parser.rb: test to demonstrate leak.

* ext/psych/parser.c: set parser encoding based on the YAML input
  rather than user configuration.
* test/psych/test_encoding.rb: corresponding tests.
* test/psych/test_parser.rb: ditto
* test/psych/test_tainted.rb: ditto

* ext/psych/parser.c: removed external encoding setter, allow parser
  to be reused.
* ext/psych/lib/psych/parser.rb: added external encoding setter.
* test/psych/test_parser.rb: test parser reuse

* ext/psych/lib/psych/visitors/to_ruby.rb: Added support for loading
  subclasses of String with ivars
* ext/psych/lib/psych/visitors/yaml_tree.rb: Added support for dumping
  subclasses of String with ivars
* test/psych/test_string.rb: corresponding tests

* ext/psych/lib/psych/visitors/to_ruby.rb: Added ability to load array
  subclasses with ivars.
* ext/psych/lib/psych/visitors/yaml_tree.rb: Added ability to dump
  array subclasses with ivars.
* test/psych/test_array.rb: corresponding tests

* ext/psych/emitter.c: fixing clang warnings. Thanks Joey!

* ext/psych/lib/psych/visitors/to_ruby.rb: BigDecimals can be restored
  from YAML.
* ext/psych/lib/psych/visitors/yaml_tree.rb: BigDecimals can be dumped
  to YAML.
* test/psych/test_numeric.rb: tests for BigDecimal serialization

* ext/psych/lib/psych/scalar_scanner.rb: Strings that look like dates
  should be treated as strings and not dates.

* test/psych/test_scalar_scanner.rb: corresponding tests.

* ext/psych/lib/psych.rb (module Psych): parse and load methods take
  an optional file name that is used when raising Psych::SyntaxError
  exceptions
* ext/psych/lib/psych/syntax_error.rb (module Psych): allow nil file
  names and handle nil file names in the exception message
* test/psych/test_exception.rb (module Psych): Tests for changes.

* ext/psych/parser.c (parse): parse method can take an option file
  name for use in exception messages.
* test/psych/test_parser.rb: corresponding tests.

* ext/psych/lib/psych.rb: remove autoload from psych
* ext/psych/lib/psych/json.rb: ditto

* ext/psych/lib/psych/tree_builder.rb: dump complex numbers,
  rationals, etc with reference ids.
* ext/psych/lib/psych/visitors/yaml_tree.rb: ditto
* ext/psych/lib/psych/visitors/to_ruby.rb: loading complex numbers,
  rationals, etc with reference ids.
* test/psych/test_object_references.rb: corresponding tests

* ext/psych/lib/psych/scalar_scanner.rb: make sure strings that look
  like base 60 numbers are serialized as quoted strings.
* test/psych/test_string.rb: test for change.

* ext/psych/parser.c: remove unused variable.

* ext/psych/lib/psych/syntax_error.rb: Add file, line, offset, and
  message attributes during parse failure.
* ext/psych/parser.c: Update parser to raise exception with correct
  values.
* test/psych/test_exception.rb: corresponding tests.

* ext/psych/parser.c (parse): Use context_mark for indicating error
  line and column.

* ext/psych/lib/psych/scalar_scanner.rb: use normal begin / rescue
  since postfix rescue cannot receive the exception class. Thanks
  nagachika!