Feature #2565

adding hooks for better tracing

Added by Yuki Sonoda about 2 years ago. Updated 9 days ago.

[ruby-core:27448]
Status:Assigned Start date:01/06/2010
Priority:Low Due date:
Assignee:Aaron Patterson % Done:

0%

Category:core
Target version:2.0.0

Description

Hi,

I made a commit that embeded dtrace probes into Ruby so that you can
profile a Ruby application at runtime. (r26235)

Adding probes had been approved by a Ruby developer's meeting,
however, the commit was little larger than what other committers
expected. I got some objection for the commit. [ruby-dev:39954]
In the end, I decided to temporarily revert the commit. (r26243)

I discussed how we should support dynamic runtime tracing, with ko1,
mame, naruse, unak and shyouhei. The problems of the commit were:
* the probes duplicated with the event_hook framework
(rb_add_event_hook, Kernel#set_trace_func)
* Design of the probes were not verified enough.
  * more trial and error are necessary, to make it clear what is
necessary to trace a Ruby application.

I accepted ko1's suggestion:
* reverting the commit
* adding some hooks for rb_add_event_hook().
* implementing probes for dynamic runtime tracing on the event_hook framework.
  * these probes can be implemented as a gem
  * I will aget a chance for trial and error.
  * The probes possibly will be merged into Ruby itself after enough
designed and getting enough use cases.

Here is a patch to add the hooks I and ko1 talked about. (attached)
And here is an extension library that provides prove points to dtrace,
on top of the hooks. (http://github.com/yugui/vm_probes )

What do you think?  Can I commit the patch I attached?

Thank you,
-- Yuki Sonoda (Yugui)

Attachment: adding-hooks.patch

probe_hacks.patch - hacking probes in to trunk (3.2 kB) Aaron Patterson, 06/30/2011 05:41 am


Related issues

related to ruby-trunk - Feature #1279: Add DTrace Probes Closed 03/13/2009

History

Updated by Yusuke Endoh about 2 years ago

Hi,

2010/1/6 Yugui <yugui@yugui.jp>:
> What do you think?  Can I commit the patch I attached?

All the event types you added are C-level events which set_trace_func can
not monitor.  But I think that RUBY_EVENT_RESCUE may be Ruby-level event
because it is a language-level event like RUBY_EVENT_RAISE.

In addition, the modification of thread_reset_event_flags is just bug fix.
So, in advance, the part should be committed as another commit.

-- 
Yusuke ENDOH <mame@tsg.ne.jp>

Updated by Yusuke Endoh almost 2 years ago

Hi Yugui,

2010/1/6 Yugui <yugui@yugui.jp>:
> I made a commit that embeded dtrace probes into Ruby so that you can
> profile a Ruby application at runtime. (r26235)
>
*snip*
>
> Here is a patch to add the hooks I and ko1 talked about. (attached)
> And here is an extension library that provides prove points to dtrace,
> on top of the hooks. (http://github.com/yugui/vm_probes )
>
> What do you think? ?Can I commit the patch I attached?


Any update here?  I'm not against the feature.
We must determine whether 1.9.2 will aim to include the feature or
not.

Rocky Bernstein offered two suggestions.  [ruby-core:27449]
The first suggestion seems reasonable, but will take some time to
discuss the API.
I guess it is better to leave it to 1.9.3, unfortunately.

-- 
Yusuke ENDOH <mame@tsg.ne.jp>

Updated by Kazuhiro NISHIYAMA almost 2 years ago

  • Category set to core
  • Status changed from Open to Assigned
  • Assignee set to Yuki Sonoda
  • Target version set to 2.0.0
  • Start date set to 01/06/2010

Updated by Mark Porter over 1 year ago

Any update on this issue? Will we see for 1.9.3? 

Updated by Aaron Patterson 8 months ago

Hi, I am interested in this functionality too. I'm interested in this because I have an application where object creation has increased. The increased objects are hash, array, and string literals. The existing trace methods will not be triggered on allocation of those objects, and the scripting / filtering / speed of dtrace is useful to me. I've been patching trunk with the attached patch in order to debug my current issues. I don't think it's as complete as yugui's commit though. What can I do to get dtrace support in trunk? How can I help?

Updated by Mark Porter 5 months ago

I cant figure out for the life of me why this issue is not getting more traction. We're seeing more and more environments getting dtrace (nodejs now has dtrace probes) and ruby has removed them, seemingly never to return. We've got people willing to help (Aaron Patterson!!), help us help Ruby. Please?

Updated by Yui NARUSE 5 months ago

Mark's concern, why Ruby doesn't include DTrace support, is reasonable, I explain why. It is because tracing is so important that we can't implement it without careful design. But we know such too careful way annoys you. So we are considering that 2.0 (not 1.9.3) includes experimental tracing support.

Updated by Aaron Patterson 9 days ago

  • Assignee changed from Yuki Sonoda to Aaron Patterson
Since I really want this feature, I should update this ticket with my progress. I've ported the Joyant probes to trunk (and added a few). You can view the progress I've made here: https://github.com/tenderlove/ruby/tree/probes I use this branch on a daily basis with no noticeable performance penalties, but I have a few tasks remaining before I want to call my work "done". 1. Probe stability declarations One concern that ko1 had was people relying on DTrace probe APIs. DTrace allows us to declare the stability of any particular probe. This allows us to tell DTrace consumers how much they should rely on that particular API. I like the API of the probes I've added (and the Joyant probes), so I don't want to change them. However, it may be useful to declare them as unstable until we've had multiple ruby releases that contain the probes (and the community is happy with the API). 2. Tests I've started writing tests for the probes, but there is a challenge. DTrace can only be enabled by a user with elevated permissions, so the tests must be run with sudo. It's easy enough to write tests that will only execute when the user has the correct permissions, but I'm not sure what we should do about CI. 3. Autotools generating probes.h I don't understand autotools very well, so I need help with this task. At rubyconf, nobu helped me patch Makefile.in to generate the probes.h file, but it doesn't seem to automatically generate on my system. I'm not entirely sure what the problem is. https://github.com/tenderlove/ruby/commit/3f645ce71e29859256063265cfd03ffd52d38dd8 When I run `rm probes.h; make`, the probes.h file is not regenerated. I don't understand what is wrong. Any help would be greatly appreciated. That's all the updates I have for now!

Also available in: Atom PDF