Feature #14276: Ruby core debugger API

• Status: Open
• Assignee: -
• Target version: -

[dsferreira (Daniel Ferreira)]

I would like to propose the creation of a ruby core minimal API for a Debugger.

Historically we have been relying on gems as providers of debug functionality.
This has caused problems with debugger APIs being broken and the debugger gems being outdated by ruby releases.
This happened in the transitions from 1.8.* to 1.9.* and 1.9.* to 2.**

Currently ruby community relies mostly on byebug gem (+ pry) for debugging purposes.

Recently a new bug 14226 was raised due to changes in ruby internals that broke byebug.

I consider that the debugger should be a core part of a language like ruby.
Developing production code without an available debugger is not very manageable.

For this reason I believe we should have a way of guaranteeing that each ruby release will not break the debugger.

Also having full control of the debugger API is very convenient for future purposes.
This way third party entities can rely on it to develop productivity tools.

I see two ways we can accomplish this:

• Defining core Debug API namespace where we would create the minimal interface for ruby core debug functionality (Not sure if only cruby or a broader API adopted and designed in collaboration with the other ruby implementations).
• Creating debug gem in standard lib with the same purpose. Having the guarantee that the unit tests will run for every ruby release.

The main goal in my mind would be to have a bare minimum (TDD?) development environment available upon ruby installation
with each bare minimum API providing the base functionality to allow the development of full fledged solutions capable of rivaling with the best solutions in the market.
For that goal I would include here for consideration the same approach for other functionalities such as:

• Coverage
• Documentation
• Benchmarks
• Tests
• Performance
• Linter
• ?

I believe we have currently a good situation with regards to:

• Documentation (Rdoc)
• Tests (Minitest)

Why not try to replicate those examples for the remaining areas?
Are documentation and tests so much different from the rest?

If we succeeded to do that we could present ruby to the new comer with a simpler development environment easing the entry level.
We could put simple documentation in ruby-lang.org to show how to achieve a professional development workflow with ruby with no fuss.

A full TDD environment available upon installation it is a great marketing headline in my perspective.

Just to finalise.
The goal here is not to have ruby core replacing the existing gems.
The goal is to give a clear path for these gems while allowing others to develop even further in top of a sound base
without worries regarding future support on the APIs being used.

[duerst (Martin Dürst)]

• Related to Bug #14226: Revert line number of "return", "b-return" and "end" events added

[shevegen (Robert A. Heiler)]

Developing production code without an available debugger
is not manageable.

This appears to be somewhat opinionated since I do just
fine without a debugger (except for pp, just as tenderlove
once said that he is a puts debugger, I am a pp debugger).

However had, that is not to say that I necessarily disagree
with you. If possible it would be fine to rely on a more
stable API.

If we succeeded to do that we could present ruby to the
new comer with a simpler development environment easing
the entry level.

Here you make the assumption that, all the things included
and managed, automatically make ruby simpler and thus easier.

But in reality, some suggestions also work in an orthogonal
manner - having to know how testing works, such as making
minitest a mandatory part, is another barrier for newcomers.
So is having to know rdoc; I still don't know how to avoid
rdoc from making certain assumptions leading to strangely
formatted autogenerated documentation.

That is also not to say that documentation is not important,
quite the opposite. It is very important. I started to use
autogenerated README.md files a while ago, and so far it
works quite fine as entry-level documentation.

We could put simple documentation in ruby-lang.org to
show how to achieve a professional development
workflow with ruby with no fuss.

Who or what defines a "professional development workflow"?

And can this be extrapolated to other people?

Matz once said in an interview that people are different.
It's one reason why ruby is so flexible - you can adjust
it to many different use cases -> http://www.artima.com/intv/ruby.html

I am not necessarily against the proposal, mind you. I
think your core statements made are perfectly fine.

But there is not really "one true way" in ruby alone.

What may be possible is some sort of tutorial or more tutorial
on the ruby webpage or a link from it; someone may have to
write the content of what constitutes that particular
workflow that you may want to have first.

I agree in the sense that ruby should also focus and help attract
newcomers, without making things worse for the older ruby hackers.

A full TDD environment available upon installation it
is a great marketing headline in my perspective.

Ruby should not be pushed into a particular philosophy "just
like that". What about those ruby hackers who, for any reason,
do not use TDD? We could just as well say that all frameworks
must use the MVC pattern and that everyone has to use agile
and vim users write better ruby code than emacs users.

I agree with your core statement about debuggable APIs.

I assume that the ruby core team may need more information
how people use (certain) debuggers and what are the main
reasons why the debuggers break/broke. Whether it were
syntax changes or C-internal reasons that broke extensions.

[dsferreira (Daniel Ferreira)]

Developing production code without an available debugger is not manageable.

This appears to be somewhat opinionated since I do just fine without a debugger

OK. Lets replace the wording from "not manageable" to "not very manageable"?

Here you make the assumption that, all the things included
and managed, automatically make ruby simpler and thus easier.

Having the tools by default upon ruby installation gives a ready environment to work with.
The tools would be optional and only available using require.
From that I don't see where is the extra complexity to the new comer.

Ruby should not be pushed into a particular philosophy "just like that".

It was just a serious of examples. Not my intention to make ruby biased towards a particular philosophy.
I totally agree that ruby should be open to multiple philosophies.

I tried to emphasise some reasons to support my idea of having some core tools.
I presented a list that could be smaller or bigger.
It is up to Matz and the community to decide what best fits.

[dsferreira (Daniel Ferreira)]

• Description updated (diff)

[duerst (Martin Dürst)]

dsferreira (Daniel Ferreira) wrote:

Developing production code without an available debugger is not manageable.

This appears to be somewhat opinionated since I do just fine without a debugger

OK. Lets replace the wording from "not manageable" to "not very manageable"?

I also work fine without a debugger except maybe once per year, and then it's usually for C, not Ruby. But I know some people who use debuggers extensively, and we shouldn't exclude them from Ruby.

Apart from that, do you (or anybody else) have some specific ideas or requirements for a debugging API?

[deivid (David Rodríguez)]

Hi! So there are two different proposals here. Regarding the first one:

Defining core Debug API namespace where we would create the minimal interface for ruby core debug functionality (Not sure if only cruby or a broader API adopted and designed in collaboration with the other ruby implementations).

It's already there! A bit of context:

• MRI 1.8 debugger, called ruby-debug, needed to hack into core sources extensively, and that caused the debugger to be broken in MRI 1.9.

• As a result, a fork called debugger was reworked to hack again into MRI 1.9 sources, and brought debugging support for MRI 1.9.

• ruby-core realized that this was an undesirable situation and implemented the TracePoint and the Debug Inspector APIs for debuggers (and other tools) to use. They were released with MRI 2.0 and, of course, broke debugger.

• byebug was then born using these APIs and the situation has been quite stable since. Initial MRI releases (2.0, 2.1, 2.2) surfaced several bugs in the APIs that were gradually fixed and backported. As of 2.3.0 there were no more outstanding bugs in the APIs that I know of.

• It is true that MRI 2.5.0 caused some breakage in byebug but it was nothing like the old times where the debugger wouldn't even compile. In this case, only a specific method calculating the potential breakpoint locations needed to be patched. There was also some intentional changes in the APIs done for performance reasons.

So, I'd say we're fine since ruby has had a stable debugger for 5 major releases now. :)

[dsferreira (Daniel Ferreira)]

Thanks David for your input.

Very good resume of the situation which will help a lot on the ticket.
And also it is awesome to have you here discussing the subject with us.

I was planning to reply to Martin pretty soon about his questions but your input is speeding it up to not lose momentum.

...implemented the TracePoint and the Debug Inspector APIs for debuggers (and other tools) to use.

If I'm not mistaken those APIs are C APIs?

What I would like to see would be a Ruby API in ruby core if possible under the namespace Debug with all things "Debug" under that namespace.

require "debug" would bring the debugger up with the minimal functionality. A selected set of the functionality provided by byebug.

Like if it was byebug-base and byebug/ext/byebug would not be needed anymore.

The Debug namespace I believe should be part of ruby core. It is a must for me.

Does this make sense?

[dsferreira (Daniel Ferreira)]

Sorry David I was afraid to be incorrect here. Wrote to fast.
Just looked into your comments regarding the C API being used in byebug.

Why are you using the C API’s instead of the ruby classes directly?

Also, what is the ruby class for the Debug Inspector?

[deivid (David Rodríguez)]

Welcome!

Regarding your other proposal:

Creating debug gem in standard lib with the same purpose. Having the guarantee that the unit tests will run for every ruby release.

I don't have a strong opinion on this actually, I'm fine with anything. But I emphasize that things are quite stable at the moment.

If I'm not mistaken those APIs are C APIs?

The TracePoint API is definitely exposed not only for C-extensions but also for ruby (https://ruby-doc.org/core-2.5.0/TracePoint.html). I don't think the other API is exposed directly but there might be ways to achieve the same stuff it provides in plan ruby, not sure.

Actually, my idea for the future of byebug is to reimplement the debugger without a C-extension (making the extension optional), and thus indirectly adding support for JRuby and maybe other implementations as well.

require "debug" would bring the debugger up with the minimal functionality. A selected set of the functionality provided by byebug.

Like if it was byebug-base and byebug/ext/byebug would not be needed anymore.

Seems like a possibility but note that the C-extension is not about providing "extra" functionality, is about providing the base functionality in a fast way. Also, I actually kind of remember a debug.rb file in ruby-core that you could require to get some bare debugging functionality... Not sure though.

Hope this helps!