Bug #20641
closed`lib/bundled_gems.rb` makes `Kernel.require` over 100x slower
Description
I just discovered this while profiling Active Record's test suite, and I noticed 40% of the runtime was in $LOAD_PATH.resolve_feature_path
, so much I thought it was a profiler bug.
But it turns out it's real. Various APIs do call require
late, for instance Psych calls require 'date'
every single time it parses a date: https://github.com/ruby/psych/blob/be0ba74e5613c20f213403e15914d24944c2652d/lib/psych/scalar_scanner.rb#L64
I've put together a quick benchmark:
# frozen_string_literal: true
require "bundler/inline"
gemfile(true) do
source "https://rubygems.org"
gem "benchmark-ips"
end
Benchmark.ips do |x|
x.report(RUBY_VERSION) { require "erb" }
x.save! "/tmp/require.bench"
x.compare!
end
And the difference is massive:
Calculating -------------------------------------
3.2.2 6.450M (± 1.2%) i/s - 32.822M in 5.089538s
Comparison:
3.2.2: 6449899.4 i/s
3.3.3: 46996.8 i/s - 137.24x slower
And that's with a small $LOAD_PATH
, the bigger the application, the worse it is, if I add:
100.times do |i|
$LOAD_PATH.unshift("/tmp/empty-#{i}")
end
Calculating -------------------------------------
3.3.3 6.198k (± 6.1%) i/s - 30.968k in 5.018955s
Comparison:
3.2.2: 3380939.6 i/s
3.3.3: 6198.4 i/s - 545.46x slower
I'm looking at a way to speed this up, but this is big enough that I believe we should backport the fix.
Updated by byroot (Jean Boussier) 5 months ago
Some extra context. This codepath is only active if using Bundler >= 2.5.0.
Updated by byroot (Jean Boussier) 5 months ago
- Status changed from Open to Closed
Applied in changeset git|82aee1a9467c0f1bd33eb0247c5a0a8b8b9a5049.
bundled_gems.rb: Add a fast path
[Bug #20641] Gem::BUNDLED_GEMS.warning?
adds a lot of extra
work on top of require
. When the call end up atually loading code
the overhead is somewhat marginal.
However it's not uncommon for code to go some late require
in some
paths, so it's expected that calling require
with something already
required is somewhat fast, and bundled_gems.rb
breaks this assumption.
To avoid this, we can have a fast path that in most case allow to
short-circuit all the heavy computations. If we extract the feature
basename and it doesn't match any of the bundled gems we care about
we can return very early.
With this change require 'date'
is now only 1.33x slower on Ruby
3.3.3, than it was on Ruby 3.2.2, whereas before this change it
was at least 100x slower.
Updated by k0kubun (Takashi Kokubun) 3 months ago
- Backport changed from 3.1: DONTNEED, 3.2: DONTNEED, 3.3: REQUIRED to 3.1: DONTNEED, 3.2: DONTNEED, 3.3: DONE
ruby_3_3 4667f8ec10269b0b5deca459f098abbdf3bae4ec.