Bug #17101: YAML.load_file: Massive slowdown under Ruby 2.7 vs. Ruby 2.4

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x64-mingw32]
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN

[fitmap (Justin Peal)]

When use Ruby 2.4, the following program runs fast, After ungrade to Ruby 2.7, the same program runs very very slow. Please unzip the attach first for test.

require 'yaml'
YAML.load_file('qlnv_h_h.yaml')

[MSP-Greg (Greg L)]

You used the phrase 'runs very very slow'. I found 2.7 slower, as below. What were the times you had?

Using Psych.safe_load, got the following for several mingw builds, along with an mswin master build.

All returned a hash with 608 keys. Tests done on an older slow notebook.

Interesting that the mswin build was much faster than all the others. I should run some of the Ruby benchmarks, as that was unexpected.

Time  Psych  Ruby
 19   2.2.2  2.4.10
 41   3.0.2  2.5.8
 38   3.1.0  2.6.6
 26   3.1.0  2.7.1
 25   3.1.0  2.8.0dev (2020-08-03T10:46:39Z master 1d637b1f5e) [x64-mingw32]
 12   3.1.0  2.8.0dev (2020-08-03T10:46:39Z master 1d637b1f5e) [x64-mswin64_140]

[S_H_ (Shun Hiraoka)]

I tried this benchmark in several ruby's(Ubuntu18.04 in WSL2. using rbenv).

prelude: |
  require 'yaml'
benchmark: 
  load_file: YAML.load_file('qlnv_h_h.yaml')
loop_count: 1

And, result.

sh@MyComputer:~/rubydev/bugs$ benchmark-driver yml.yaml --output compare --rbenv '2.4.4;2.5.8;2.6.6;2.7.1;2.8.0-dev'
Calculating -------------------------------------
                          2.4.4       2.5.8       2.6.6       2.7.1   2.8.0-dev 
           load_file      0.113       0.075       0.076       0.058       0.134 i/s -       1.000 times in 8.839451s 13.406090s 13.240691s 17.256679s 7.435239s

Comparison:
                        load_file
           2.8.0-dev:         0.1 i/s 
               2.4.4:         0.1 i/s - 1.19x  slower
               2.6.6:         0.1 i/s - 1.78x  slower
               2.5.8:         0.1 i/s - 1.80x  slower
               2.7.1:         0.1 i/s - 2.32x  slower

Between 2.5 and 2.7 is slow, but master(2.8.9-dev) is very fast(faster than 2.4!).

And checked yaml versions(in these ruby's).

Ruby version	YAML version
2.8.0-dev	0.2.5
2.7.1	0.2.1
2.6.6	0.2.1
2.5.8	0.1.7
2.4.4	0.1.7

[fitmap (Justin Peal)]

I run this program on Windows 10 64-bit. Very very slow: I wait more than 10 minutes, then I press Ctrl-Break. Now I install Ruby 2.6, then the program runs as fast as Ruby 2.4.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

I tried this on OpenBSD, and ruby 2.7 and 3.0 were faster than ruby 2.4 (all of these use libyaml 0.2.2):

/usr/local/bin/ruby27:         0.1 i/s
/usr/local/bin/ruby30:         0.1 i/s - 1.00x  slower
/usr/local/bin/ruby24:         0.0 i/s - 1.46x  slower
/usr/local/bin/ruby26:         0.0 i/s - 1.83x  slower
/usr/local/bin/ruby25:         0.0 i/s - 1.98x  slower

I also tested on Windows, where performance on ruby 3.0 and 2.7 is not as good as 2.4, but still much better than 2.5 and 2.6 (these use whatever libyaml is embedded in RubyInstaller):

C:\\ruby24-x64\\bin\\ruby:         0.1 i/s
C:\\ruby30-x64\\bin\\ruby:         0.0 i/s - 1.26x  slower
C:\\ruby27-x64\\bin\\ruby:         0.0 i/s - 1.30x  slower
C:\\ruby26-x64\\bin\\ruby:         0.0 i/s - 2.06x  slower
C:\\ruby25-x64\\bin\\ruby:         0.0 i/s - 2.13x  slower

It's likely the decrease in performance in Windows on ruby 2.7 and 3.0 is due to different versions of libyaml:

D:\>c:\Ruby30-x64\bin\ruby -r yaml -e "p Psych::LIBYAML_VERSION"
"0.2.5"

D:\>c:\Ruby27-x64\bin\ruby -r yaml -e "p Psych::LIBYAML_VERSION"
"0.2.2"

D:\>c:\Ruby26-x64\bin\ruby -r yaml -e "p Psych::LIBYAML_VERSION"
"0.2.1"

D:\>c:\Ruby25-x64\bin\ruby -r yaml -e "p Psych::LIBYAML_VERSION"
"0.1.6"

D:\>c:\Ruby24-x64\bin\ruby -r yaml -e "p Psych::LIBYAML_VERSION"
"0.1.6"

There no longer appears to be a massive slowdown, and it appears that for the same libyaml version, ruby 2.7 and 3.0 are faster, so I'm going to close this.