Feature #17287: Faster Pathname FileUtils methods

• Status: Open
• Priority: Normal
• Assignee: -
• Target version: -

[schneems (Richard Schneeman)]

I have a patch that I would like to merge into Pathname for increased performance. I understand that akr maintains pathname and may not be on GitHub. Here is a link to my patch:

https://github.com/ruby/ruby/pull/3693

Here is the diff:

$ git diff master
diff --git a/ext/pathname/lib/pathname.rb b/ext/pathname/lib/pathname.rb
index e6fb90277d..c3af24837f 100644
--- a/ext/pathname/lib/pathname.rb
+++ b/ext/pathname/lib/pathname.rb
@@ -575,12 +575,13 @@ def find(ignore_error: true) # :yield: pathname


 class Pathname    # * FileUtils *
+  autoload(:FileUtils, 'fileutils')
+
   # Creates a full path, including any intermediate directories that don't yet
   # exist.
   #
   # See FileUtils.mkpath and FileUtils.mkdir_p
   def mkpath
-    require 'fileutils'
     FileUtils.mkpath(@path)
     nil
   end
@@ -591,7 +592,6 @@ def mkpath
   def rmtree
     # The name "rmtree" is borrowed from File::Path of Perl.
     # File::Path provides "mkpath" and "rmtree".
-    require 'fileutils'
     FileUtils.rm_r(@path)
     nil
   end

Description

Currently when calling any of the "FileUtils" methods on pathname require is called every time even though that library might already be loaded. This is slow.

We can speed it up by either checking first if the constant is already defined, or by using autoload.

Using defined speeds up the action by about 300x and using autoload is about twice as fast as that (600x faster than current require method).

I'm proposing we use autoload:

require 'benchmark/ips'

Benchmark.ips do |x|
  autoload(:FileUtils, "fileutils")
  x.report("require") { require 'fileutils' }
  x.report("defined") { require 'fileutils' unless defined?(FileUtils) }
  x.report("autoload") { FileUtils }

  x.compare!
end

# Warming up --------------------------------------
#              require     3.624k i/100ms
#              defined     1.465M i/100ms
#             autoload     2.320M i/100ms
# Calculating -------------------------------------
#              require     36.282k (± 2.4%) i/s -    184.824k in   5.097153s
#              defined     14.539M (± 2.0%) i/s -     73.260M in   5.041161s
#             autoload     23.100M (± 1.9%) i/s -    115.993M in   5.023271s

# Comparison:
#             autoload: 23099779.2 i/s
#              defined: 14538544.9 i/s - 1.59x  (± 0.00) slower
#              require:    36282.3 i/s - 636.67x  (± 0.00) slower

Because this autoload is scoped to Pathname it will not change the behavior of existing programs that are not expecting FileUtils to be loaded yet:

ruby -rpathname -e "class Pathname; autoload(:FileUtils, 'fileutils'); end; puts FileUtils.exist?('foo')"
Traceback (most recent call last):
-e:1:in `<main>': uninitialized constant FileUtils (NameError)