Project

General

Profile

Actions

Feature #12009

closed

[PATCH] psych: pre-freeze string keys for hashes

Added by normalperson (Eric Wong) about 8 years ago. Updated about 6 years ago.

Status:
Rejected
Target version:
-
[ruby-core:72950]

Description

With the following example, this reduces allocations from 346 to 324
strings when calling Psych.load on a 26-entry hash:


require 'psych'
require 'objspace'
before = {}
after = {}
str = [ '---', *(('a'..'z').map { |k| "#{k * 11}: 1" }), '' ].join("\n")
GC.disable
ObjectSpace.count_objects(before)
h = Psych.load(str)
ObjectSpace.count_objects(after)
p(after[:T_STRING] - before[:T_STRING])

Allocating 324 strings for 26 hash keys is still expensive. More
work will be needed to reduce allocations further...

Tested on x86-64.

It would actually be awesome if Psych could use the fstring table
and have string keys auto-deduped as in Ruby source code.


Files

Updated by nobu (Nobuyoshi Nakada) about 8 years ago

  • Description updated (diff)
  • Status changed from Open to Assigned

Updated by normalperson (Eric Wong) about 8 years ago

Alternative patch which is stronger to help the case where I have
arrays of hashes; each hash having the same set of keys:

http://80x24.org/spew/20160120035342.20168-1-e@80x24.org/raw

However, this does not help the case where I have multiple
Psych.load invocations...

Unsubscribe:
http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core

Updated by normalperson (Eric Wong) about 6 years ago

  • Status changed from Assigned to Rejected

To be solved in a more general manner [Feature #14225].

(Partially solved for untainted data in [Feature #13725])

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0