Bug #17389: New docs for non-blocking Fibers and scheduler

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN

[zverok (Victor Shepelev)]

GitHub PR: https://github.com/ruby/ruby/pull/3891

Copying from its description:

I propose new documentation approach for new features in Ruby 3.0: non-blocking Fiber and the scheduler.

Right now, the documentation is in a confusing state. First, the doc/scheduler.md is even not rendered correclty: https://docs.ruby-lang.org/en/master/doc/scheduler_md.html (relatively easy to fix: wrong markdown markup for the code-blocks), and Fiber class itself has no mention for new concepts, and no docs for new methods: https://docs.ruby-lang.org/en/master/Fiber.html#method-c-schedule.

But on the bigger level, the documentation is quite hard to follow unless you are already fully in context of asynchronous loops and schedulers.

I am trying to fix it by:

• adding to the Fiber class necessary docs, both high-level overview and particular method details
• redocumenting the expected scheduler interface via "imaginary" Fiber::SchedulerInterface class: it is present only in docs to leverage RDoc's method-by-method documentation, be able to link to them separately and so on

Test rendering of the docs on my personal site:

• Fiber
• Fiber::SchedulerInterface

I have not yet in this PR removed doc/scheduler.md, but I suggest to, as it is completely superseded by new docs.

I'd be really grateful if @ioquatix (Samuel Williams) will find some time to review this initiative in the upcoming days.

[zverok (Victor Shepelev)]

@matz (Yukihiro Matsumoto), can you please look into this question?

@ioquatix (Samuel Williams) have updated scheduler.md→fiber.md in https://github.com/ruby/ruby/pull/3965/files, where it now looks like a generic Fiber class docs, but I still believe the docs should be in the class itself, and that Fiber::SchedulerInterface is the right way to document how the scheduler is expected to work, which I propose in https://github.com/ruby/ruby/pull/3965/files and this ticket.

TBH, I feel completely lost in regards to who's responsible for the documentation (besides each feature's author) and who should/want/can make decisions about it.

[zverok (Victor Shepelev)]

@ioquatix (Samuel Williams) @matz (Yukihiro Matsumoto) @marcandre (Marc-Andre Lafortune) @nobu (Nobuyoshi Nakada) @mame (Yusuke Endoh) Is this OK that 3.0 will be released without proper docs for one of the major new features?..

[hakusaro (Lucas Nicodemus)]

I reckon that Samuel is just wrapped up in some other things. Async is the gem carrying the "example torch," and he's been putting in work on native scheduler use. He's also done more than a few conference talks at Ruby Kaigi and published several reports on Ruby concurrency. I'd say that while the Ruby text documentaton isn't quite on par with the rest of the material, the Fiber changes are far from undocumented.

See:

• Fibers are the right solution (talk) and text version.
• Progress report 1 and progress report 2 on the concurrency project for the 2019 Ruby Association Grant.
• Asynchronous ruby

In general, this is really hard CS stuff for the uninitiated (at least, it is/was to me). I completely sympathize with you, but I feel like there's just a lot of dense material about this required to understand it.

I think it'd be worth explaining components of your PR and where they line up with the latest changes (particularly, why/what the end goal is). Some kind of comprehensive scheduling/concurrency Ruby documentation, between Fibers + Ractors, is probably necessary in the long run. But the people who are going to be working with these tools right out the gate are also like to be people who are already experienced in this field(?).

Given all this, it may not be super urgent -- but I tend to agree that anything that helps understanding is beneficial.

[zverok (Victor Shepelev)]

@hakusaro (Lucas Nicodemus) This all would be (somewhat) true if I'd demanded "do document that new feature".
But I did it myself, I just asked if somebody can review the docs (done since my last comment by @marcandre (Marc-Andre Lafortune), huge thanks!)

Overall, I don't think the "everyone is too busy right now and docs aren't that important" attitude is a good thing for a major public release of the popular programming language...

Honestly, I feel like Ruby really needs some "documentation working group" to manage this aspect.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Closed