Feature #10844: TracePoint API needs an event to inform about creating/removing a new frame without calling something

• Status: Feedback
• Assignee: ko1 (Koichi Sasada)
• Target version: -

[os97673 (Oleg Sukhodolsky)]

There are some situations when Ruby VM creates a new frame even if there is no calls
E.g. ensure create a new block.
Here is a simple program which shows that the frame is added but no event is reported.
It is important to known about such situations to correctly implement stepping in debugger.
It would be nice if the api reports about such situations with something like :start_frame/:end events.
What do you think?

def actual_stack_size(frames_to_skip=3)
  bt = Thread.current.backtrace_locations(frames_to_skip)
  bt.size
end
TracePoint.new do |tp|
  printf "%8s %s:%-2d %13s %d\n", tp.event, tp.path, tp.lineno, tp.method_id, actual_stack_size
end.enable

begin
  raise
rescue
  'error'
end
1

c_return ruby/debugger/ruby-16236.rb:7         enable 1
    line ruby/debugger/ruby-16236.rb:9                1
    line ruby/debugger/ruby-16236.rb:10               1
  c_call ruby/debugger/ruby-16236.rb:10         raise 1
c_return ruby/debugger/ruby-16236.rb:10         raise 1
  c_call ruby/debugger/ruby-16236.rb:10           new 1
  c_call ruby/debugger/ruby-16236.rb:10    initialize 2
c_return ruby/debugger/ruby-16236.rb:10    initialize 2
c_return ruby/debugger/ruby-16236.rb:10           new 1
  c_call ruby/debugger/ruby-16236.rb:10     backtrace 1
c_return ruby/debugger/ruby-16236.rb:10     backtrace 1
   raise ruby/debugger/ruby-16236.rb:10               1
  c_call ruby/debugger/ruby-16236.rb:12           === 2
c_return ruby/debugger/ruby-16236.rb:12           === 2
    line ruby/debugger/ruby-16236.rb:12               2
    line ruby/debugger/ruby-16236.rb:14               1

[ko1 (Koichi Sasada)]

• Status changed from Open to Feedback
• Assignee set to ko1 (Koichi Sasada)

I want to reject this proposal because:

(1) ensure clause can be executed without pushing frame

begin

no exception

ensure
puts "ensure"
end

The line [puts "ensure"] is called without pushing new frame.

(2) performance down

Inserting hook points slows down execution speed even if it is enabled.
We need to be careful for that.

It is important to known about such situations to correctly implement stepping in debugger.

Maybe we can provide another way.
For example, line events also help with current implementation.

[os97673 (Oleg Sukhodolsky)]

Koichi Sasada wrote:

I want to reject this proposal because:

(1) ensure clause can be executed without pushing frame

begin

no exception

ensure
puts "ensure"
end

The line [puts "ensure"] is called without pushing new frame.

not sure how it is related :( The even is supposed to be filed only if a new frame has been pushed so it is ok to not file it here.

(2) performance down

Inserting hook points slows down execution speed even if it is enabled.
We need to be careful for that.

I agree, the performance is always important.

It is important to known about such situations to correctly implement stepping in debugger.

Maybe we can provide another way.
For example, line events also help with current implementation.

I'm not sure I understand how it may help here :( debugger still needs to known size of frame's stack.
I.e. if at the previous stop (e.g. breakpoint) we had X frames then next (step over) is supposed to stop execution
an text line event which frames stack's size is X or less.
Consider the following code

begin
  raise
rescue
  puts '1'
end
puts '2'

If debugger is stopped at "raise" line most users expect that after "next"(step over) command it will stop at "puts '1'"
And currently to implement such behavior both byebug and debase have to count frame's stack size manually :(

[deivid (David Rodríguez)]

@Oleg What's the problem with counting frame's stack size manually?

[os97673 (Oleg Sukhodolsky)]

David Rodríguez wrote:

@Oleg What's the problem with counting frame's stack size manually?

First: it introduced one more notion: calculated stack size, which we should use for stepping only.
Second: its calculation is not that simple you not only need to inc/dec it but should also compare it with real stack size
to fix errors you may have due to absence of new/end frame events :(
Thus I believe the API should be improved to make debugger's implementation easier.

[deivid (David Rodríguez)]

Both of those issues arised from several bugs in the TracePoint API that led to unbalanced counts of call/return events in some cases. Once all those bugs are fixed, the calculated stack size should be reliable enough so that a single notion of stack size is need. Furthermore, I don't think Byebug is actually using the "real stack size" anymore.

[os97673 (Oleg Sukhodolsky)]

David Rodríguez wrote:

Both of those issues arised from several bugs in the TracePoint API that led to unbalanced counts of call/return events in some cases. Once all those bugs are fixed, the calculated stack size should be reliable enough so that a single notion of stack size is need.

As for counting: could you please refer those bugs?

Furthermore, I don't think Byebug is actually using the "real stack size" anymore.

Yes, byebug doesn't use real stack size but it means it does not provide complete information about the stack :(

[deivid (David Rodríguez)]

Sure, most of the issues here were related to unbalanced counts of events: https://github.com/deivid-rodriguez/byebug/issues/16#issuecomment-47673108 All of them are already fixed. The only bug yet to be fixed related to this would be this one: https://bugs.ruby-lang.org/issues/10724, https://github.com/deivid-rodriguez/byebug/issues/103

Yes, byebug doesn't use real stack size but it means it does not provide complete information about the stack :(

Not sure what you mean here. The stack size at any moment in the execution of a thread is a single positive integer, which can be right or wrong. If this number is the correct number, I don't know how it can be more "complete". Can you be more specific and possibily open an issue in byebug with whatever is wrong/missing?

[os97673 (Oleg Sukhodolsky)]

David Rodríguez wrote:

Yes, byebug doesn't use real stack size but it means it does not provide complete information about the stack :(

Not sure what you mean here. The stack size at any moment in the execution of a thread is a single positive integer, which can be right or wrong. If this number is the correct number, I don't know how it can be more "complete". Can you be more specific and possibily open an issue in byebug with whatever is wrong/missing?

e.g. byebug is unable to detect additional frame added by rescue thus it will not be able to provide information about the frame.

[deivid (David Rodríguez)]

After rereading the whole thing, I'm still not clear about the issue...

[os97673 (Oleg Sukhodolsky)]

David Rodríguez wrote:

After rereading the whole thing, I'm still not clear about the issue...

What exactly is unclear? Why I suggest add a new event or when byebug could miss a frame?

[deivid (David Rodríguez)]

Both.

[os97673 (Oleg Sukhodolsky)]

David Rodríguez wrote:

Both.

Ok, so let's start with byebug. When it is stopped inside rescue clause (as far as I understand) it doesn't know about the frame "rescue" added because there is no event about it. Am I right? If so how it will report data local to the frame?

[deivid (David Rodríguez)]

I'm not able to try this right now but I'm pretty sure that just works in byebug... I'll confirm as soon as possible.

[deivid (David Rodríguez)]

Don't see anything wrong...

~/Work/byebug $ cat backtrace.rb
begin
  Int('hola')
rescue
  local = 17
  byebug
  puts local
end

~/Work/byebug $ ruby -Ilib -rbyebug backtrace.rb
(byebug) bt
-->  rescue in <main> at .../Work/byebug/backtrace.rb:6
       <main> at .../Work/byebug/backtrace.rb:1
(byebug) local
17