Project

General

Profile

Actions

Bug #14372

closed

Memory leak in require with Pathnames in the $LOAD_PATH in 2.3/2.4

Added by jrafanie (Joe Rafaniello) over 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.4.3p205 (2017-12-14 revision 61247) [x86_64-darwin16]
[ruby-core:84919]

Description

There is a memory leak that we have found on ruby 2.3.6 and 2.4.3 that happens on Mac OSX and Linux. Ruby 2.2.6 and 2.5.0 do not leak. We have not tested other platforms.

If $LOAD_PATH contains one or more Pathname objects, require without a fully qualified path, such as require 'ostruct', will leak if you do this require many times.

For example, the following script will leak very quickly on ruby 2.3.6 and 2.4.3:

require 'pathname'

puts Process.pid
$LOAD_PATH.unshift(Pathname.new(__dir__))

dot      = "."
filename = "ostruct"
1000.times { 1000.times { require filename }; print dot; GC.start; }

From what we can understand, it appears that rb_require_internal calls rb_feature_p which ultimately calls rb_file_expand_path_fast and resizes a string. It doesn't seem like the memory is ever freed either in c or the garbage collector. This happens many times, perhaps because Pathname objects, unlike Strings, aren't cached in loaded features so they get expanded each time. See below:

https://github.com/ruby/ruby/blob/v2_3_6/load.c#L43-L47

We can workaround this problem by converting Pathname objects in $LOAD_PATH to Strings, but this leak should be fixed since this it's common to use Pathname objects in $LOAD_PATH.

We used the instruments tool on OSX to show one such leak and the callstack, see attached image.


Files

Instruments2 2018-01-18 11-43-32.png (181 KB) Instruments2 2018-01-18 11-43-32.png jrafanie (Joe Rafaniello), 01/18/2018 04:44 PM

Related issues

Related to Ruby master - Bug #10222: require_relative and require should be compatible with each other when symlinks are usedClosedActions
Actions

Also available in: Atom PDF