Project

General

Profile

Feature #10344

[PATCH] Implement Fiber#raise

Added by nome (Knut Franke) almost 4 years ago. Updated 10 months ago.

Status:
Open
Priority:
Normal
Target version:
-
[ruby-core:65537]

Description

While it is possible to implement this in pure Ruby (by wrapping Fiber.yield and Fiber#resume), this feels like a low-level feature that ought to be provided out of the box. Also, the C implementation is more straight-forward, and more efficient. Unfortunately, it is not quite possible to implement this as a C extension module (without resorting to wrappers again); cf. the change to make_passing_arg().

Example usage:

fib = Fiber.new do
  counter = 0
  loop { counter += Fiber.yield }
  counter
end
fib.resume
fib.resume 10
fib.resume 100
fib.raise StopIteration # => 110
0001-Implement-Fiber-raise.patch (4.12 KB) 0001-Implement-Fiber-raise.patch nome (Knut Franke), 10/09/2014 12:19 AM
0001-Implement-Fiber-raise.patch (3.51 KB) 0001-Implement-Fiber-raise.patch updated patch with some doc corrections nome (Knut Franke), 10/18/2014 01:16 PM
0001-Implement-Fiber-raise-in-ext-fiber.patch (3.6 KB) 0001-Implement-Fiber-raise-in-ext-fiber.patch variant adding Fiber#raise only in ext/fiber nome (Knut Franke), 10/18/2014 01:16 PM

History

#1 [ruby-core:65540] Updated by ko1 (Koichi Sasada) almost 4 years ago

Do you have good example to use it?

#2 [ruby-core:65558] Updated by funny_falcon (Yura Sokolov) almost 4 years ago

Yes: it is missing piece to make "greenlet" port, in other words - green threads that mimics real threads.
To accomplish that there is a need to raise exception in a fiber waiting on event if event fails.
As Knut said it could be implemented in a Ruby, but it feels to be dirty way. If implemented in C it will lead to cleaner implementation.

#3 [ruby-core:65618] Updated by nome (Knut Franke) almost 4 years ago

For some more sophisticated examples, see https://github.com/nome/coroutines. The library does work with vanilla Ruby, but the patch improves performance.

Also, similar code can be simplified by using Fiber#raise. Compare e.g. the two implementation of Consumer::Yielder#await at
https://github.com/nome/coroutines/blob/master/lib/coroutines/base.rb

#4 [ruby-core:65747] Updated by ko1 (Koichi Sasada) almost 4 years ago

On 2014/10/12 1:28, Knut.Franke@gmx.de wrote:

For some more sophisticated examples, see https://github.com/nome/coroutines. The library does work with vanilla Ruby, but the patch improves performance.

Also, similar code can be simplified by using Fiber#raise. Compare e.g. the two implementation of Consumer::Yielder#await at
https://github.com/nome/coroutines/blob/master/lib/coroutines/base.rb

I understand this feature helps some libraries. But I can't understand
why it is important.

I'm afraid that introducing such feature increases complexity of Fiber.
Basically, I want to recommend strongly that using Fiber as
semi-croutine, ristricted feature.

At least, such feature should be located at ext/fiber.

--
// SASADA Koichi at atdot dot net

#5 [ruby-core:65780] Updated by funny_falcon (Yura Sokolov) almost 4 years ago

At least, such feature should be located at ext/fiber.

even if ext/fiber it is still should be implemented in C .

I'm afraid that introducing such feature increases complexity of Fiber.

Not too much. Look at the patch: it uses same hook as rb_fiber_terminate.

By the way, why Thread has this feature?
http://www.ruby-doc.org/core-2.1.3/Thread.html#method-i-raise
And why Fiber hasn't? In fact, it is strange that this feature still not
presented, cause it is expected to exists.

#6 [ruby-core:65782] Updated by nome (Knut Franke) almost 4 years ago

I understand this feature helps some libraries. But I can't understand why it is important.

Without Fiber#raise, libraries need to use some rather inelegant and inefficient workarounds (wrapping or tagging exceptions, wrapping Fiber.yield etc). Also, having Thread#raise arguably leads one to expect the same functionality in Fiber. I'm not sure whether the feature qualifies as "important", but I still think it is worth including.

Basically, I want to recommend strongly that using Fiber as semi-croutine, ristricted feature.

I think Consumer is an example of a semi-coroutine (in the sense that it uses Fiber.yield, not Fiber#transfer) that benefits from having Fiber#raise.

At least, such feature should be located at ext/fiber.

I disagree, because the feature is applicable to the restricted (semi-coroutine) fibers available without requiring 'fiber', and indeed implementable on top of them. Nevertheless, I've attached a variant of the patch which adds the method only in ext/fiber (as a compromise solution).

#7 [ruby-core:65862] Updated by ko1 (Koichi Sasada) almost 4 years ago

Yura Sokolov wrote:

At least, such feature should be located at ext/fiber.

even if ext/fiber it is still should be implemented in C .

I agree.

I'm afraid that introducing such feature increases complexity of Fiber.

Not too much. Look at the patch: it uses same hook as rb_fiber_terminate.

I don't mention about implementation, but specification.

By the way, why Thread has this feature?
http://www.ruby-doc.org/core-2.1.3/Thread.html#method-i-raise
And why Fiber hasn't? In fact, it is strange that this feature still not
presented, cause it is expected to exists.

Thread#raise is one of inter communication method (but asynchronouse, difficult, unrecommended way).

Fibers can be under control without such communication.

#8 [ruby-core:65863] Updated by ko1 (Koichi Sasada) almost 4 years ago

Knut Franke wrote:

I understand this feature helps some libraries. But I can't understand why it is important.

Without Fiber#raise, libraries need to use some rather inelegant and inefficient workarounds (wrapping or tagging exceptions, wrapping Fiber.yield etc).

I understand.

Also, having Thread#raise arguably leads one to expect the same functionality in Fiber. I'm not sure whether the feature qualifies as "important", but I still think it is worth including.

I don't expect there is Fiber#raise. Fiber and Thread are different.

Basically, I want to recommend strongly that using Fiber as semi-croutine, ristricted feature.

I think Consumer is an example of a semi-coroutine (in the sense that it uses Fiber.yield, not Fiber#transfer) that benefits from having Fiber#raise.

Interesting. I don't know details of this library. Could you explain why it is important in this library?

#9 [ruby-core:65892] Updated by nome (Knut Franke) almost 4 years ago

Koichi Sasada wrote:

I think Consumer is an example of a semi-coroutine (in the sense that it uses Fiber.yield, not Fiber#transfer) that benefits from having Fiber#raise.

Interesting. I don't know details of this library. Could you explain why it is important in this library?

Consumer provides an abstraction for (semi-)coroutines that accept (consume) values; analogous to an Enumerator, which produces (enumerates) values. Since a consumer may need to do resource cleanup and/or produce a final result, we need some way to signal end of values. I think the most natural way to do this is to raise StopIteration in the consumer: The situation is analogous to calling Enumerator#next when no more values are available, and StopIteration terminates Kernel#loop, which often allows writing consumers very concisely.

In general, I think Fiber is a powerful primitive that can be used by libraries to build abstractions like Enumerator, Consumer and others. While Enumerator does not require the #raise feature, other abstractions can very well benefit from it.

#10 [ruby-core:82870] Updated by yugui (Yuki Sonoda) about 1 year ago

  • Target version set to 2.5
  • Assignee set to ko1 (Koichi Sasada)

Let me add another use case which I discussed with naruse and akr in RubyKaigi.

Fiber#raise is helpful to improve compatibility of methods with blocks (internal iterators) and Enumerator (external iterators).
This happened for me when I tried to write an adapter between two independently-developed APIs.

Here's a simplified example below. You can also find the real example in https://github.com/supership-jp/activerecord-spanner-adapter/blob/master/lib/active_record/connection_adapters/spanner/client.rb.

Example

Suppose that we have a third-paty API, which we cannot control.

def with_transaction
  tx = Transaction.new
  begin
    yield tx
  rescue
    tx.rollback
  else
    tx.commit
  end
end

And now I need to begin a transaction in a method and then need to commit or rollback the transaction in other methods.

class TransactionManager
  def begin_transaction
     @iter = Transaction.enum_for(:begin_transaction)
     @tx = @iter.next # skip error handlings for simplicity
  end

  def commit_transaction
    loop { @iter.next }
  ensure
    @tx = nil
  end

  def abort_transaction(reason = RuntimeError.new)
    # ??? How can I let the iterator know the error raised
    @iter.raise(reason)
  end

  attr_reader :tx
end

In other words, internal iterators in Ruby can manage resources in it but external iterators can't even if the user tried to take responsibility of calling cleanup mechanism.
In the real usecase, with_transaction was an API given by google-cloud-spanner gem and the interface of the TransactionManager was the one I had to implement for ActiveRecord.

Proposal

Although I could certainly implement the adapter without the native Fiber#raise, it would be still helpful if internal iterators and external iterators in Ruby were more consistent.

Can we have Fiber#raise and Enumerator#raise on top of it to improve the consistency? ko1?

FYI: I've confirmed that the patch by nome still works fine for ruby-trunk. https://github.com/yugui/ruby/commit/5a04a8b49b5086686fd75c6635c95c12ccc6caa8

#11 Updated by naruse (Yui NARUSE) 10 months ago

  • Target version deleted (2.5)

Also available in: Atom PDF