Misc #17050
closedprofiler gem
Description
I would like to reuse profiler gem namespace:
https://rubygems.org/gems/profiler/versions
It currently conflicts with profile/lib/profiler.rb
however this is no longer part of stdlib.
Updated by ioquatix (Samuel Williams) over 3 years ago
Also I'd be happy to receive/resuse the profile
namespace too but either one is fine and the profile
gem actually has an interesting implementation.
Updated by duerst (Martin Dürst) over 3 years ago
Can you tell us shortly what you plan to use that gem/namespace for? If it's something that can (in some reasonable way) replace the (former?) profile/profiler functionality, then please go for it. At one point, I was planning to work on this, but I didn't find the time.
Updated by ioquatix (Samuel Williams) over 3 years ago
My initial goal is to expose some general data structures for efficiently capturing large memory profiles on top of ObjectSpace
and GC
modules, similar to how memory_profiler gem works. I don't mind making separate gem, but it seems like the profile
or profiler
gem would be ideal and have not been updated for a long time. Initially I was thinking of making separate gems and trying to provide a unified interface, e.g.
-
memory_profiler
->profiler-memory
-
stackprof
->profiler-stack
-
ruby-prof
->profiler-sample
etc.
I think that's too ambitious initially, but yeah, that's a longer term idea, with an initial focus on profiler-memory
of which I've already got working code and just looking how to structure it best for longer term integration.
A sideways step would be to provide rspec-memory
hooks for using these profilers either for general summary or assertions around blocks of code (of which I already maintain the rspec-memory
gem which does some parts of this).
@ko1 (Koichi Sasada) said he wants to use profile
gem, but I'm not sure if he also intended to use profiler
gem. So, @ko1 (Koichi Sasada), can I use one of them, and you can use the other?
Updated by Eregon (Benoit Daloze) over 3 years ago
I'd recommend starting with something specific like profiler-memory
.
profiler
is very generic but therefore also unclear and will make people think it's a stdlib-like profiler, but I guess it will not be.
I also think it's too early to tell if your gem will be "the profiler everyone uses".
BTW, stackprof is a CPU sampling profiler, and ruby-prof seems a tracing CPU profiler.
As a side note it'd be nice if profiler-like gems work on more than 1 implementation.
I think most them only work on CRuby, because they use C APIs that are efficient on CRuby but basically inefficient on other implementations (see https://github.com/oracle/truffleruby/issues/2044#issuecomment-654848324 for details regarding stackprof & TruffleRuby).
Updated by ioquatix (Samuel Williams) over 3 years ago
I agree all your points. I already have working profiler-memory
to a certain extent.
Updated by Eregon (Benoit Daloze) over 3 years ago
In the profile
stdlib, both files are used: -rprofile
starts the profiler immediately, while profiler.rb
is like a library:
https://github.com/ruby/ruby/blob/ruby_2_6/lib/profile.rb
So it seems to me not advisable to reuse those "require names" for something not strictly compatible since profile
was only removed in 2.7.
Updated by byroot (Jean Boussier) over 3 years ago
@ioquatrix I’m curious. What’s the difference between your memory-profiler and the existing memory_profiler?
For context I’ve been working on https://github.com/shopify/heap-profiler for the last week, and I’m also very interested in improving ObjectSpace and GC APIs
Updated by ioquatix (Samuel Williams) over 3 years ago
I was able to profile 200+ million* allocations with my fork which was impossible with the current implementations. Right now my current plan is to upstream these changes to memory_profiler
. (*sorry, incorrectly said billions previously :)
Updated by jeremyevans0 (Jeremy Evans) over 3 years ago
- Tracker changed from Bug to Misc
- Backport deleted (
2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN)
Updated by ko1 (Koichi Sasada) over 3 years ago
- Status changed from Open to Feedback
Sorry for late response.
I have a plan to remake current profile library with recent APIs.
Updated by ioquatix (Samuel Williams) over 3 years ago
- Status changed from Feedback to Closed
Okay, I look forward to your profiler.