Bug #20081: Transfered Fiber doesn't return to Fiber that started it

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-darwin22]
• Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

[rmosolgo (Robert Mosolgo)]

Hi! I'm trying to figure out how to make sure that Fibers started with .transfer end up terminated, not just suspended. (If they're suspended, Rails thinks they're still alive, and they continue to hold onto database connections, see: https://github.com/rmosolgo/graphql-ruby/issues/4739#issuecomment-1866930914.)

So, I'm looking for way to make sure that any Fiber I start with .transfer will be properly terminated. But what I noticed is that when a transfer-based Fiber terminates, it gives control back to the top-most Fiber, not the Fiber which transfered to it. Is this intended? Here's a script to replicate the issue:

manager = Fiber.new do
  parent = Fiber.current
  worker = Fiber.new do
    puts "2. Begin Worker"
    parent.transfer
    puts "4. End Worker"
  end

  puts "1. Transfer 1"
  worker.transfer
  puts "3. Transfer 2"
  worker.transfer
  puts "5. Finished manager"
end

manager.transfer
puts "6. Finished script"

I expect the steps to print in order:

1. Transfer 1
2. Begin Worker
3. Transfer 2
4. End Worker
5. Finished manager
6. Finished script

But instead, 5. ... is skipped:

1. Transfer 1
2. Begin Worker
3. Transfer 2
4. End Worker
6. Finished script

I think that's because my worker fiber terminates and passes control back to the top-level Fiber.

Should it have passed control back to the manager? Or is there another way to make sure worker is terminated, and manager gets control?

[rmosolgo (Robert Mosolgo)]

I see that Ruby 3.3 has Fiber#kill coming (https://github.com/ruby/ruby/pull/7823), which I think will solve my problem: I can manually .kill Fibers instead of running them until they return. But still, I'm interested to learn whether this is a bug or not!

[ioquatix (Samuel Williams)]

My initial feeling is that this is not a bug.

Transferring to a fiber is a uni-directional operation and no information is expected to be kept about the caller. If that is what you desire, use resume.

There is nothing that prevents you from using resume and then transfer.

If you do desire to transfer back to a specific fiber, you can code to that, e.g.

manager = Fiber.new do
  parent = Fiber.current
  worker = Fiber.new do
    puts "2. Begin Worker"
    parent.transfer
    puts "4. End Worker"
  ensure
    # Exit:
    parent.transfer
  end

  puts "1. Transfer 1"
  worker.transfer
  puts "3. Transfer 2"
  worker.transfer
  puts "5. Finished manager"
end

manager.transfer
puts "6. Finished script"

I don't know if there is a better semantic worth trying to tease out of this, but the semantics of transfer are fairly straight forward and I'm not sure we should change it as the chance of breaking something is probably fairly high.

[rmosolgo (Robert Mosolgo)]

Thanks for taking a look! I was going from the example in the docs (https://docs.ruby-lang.org/en/master/Fiber.html#method-i-transfer), where the return value of .transfer is used by the caller.

And .transfer does return to the caller, as long as the caller is the main Fiber, for example, this prints out a sequence of messages:

parent = Fiber.current
worker = Fiber.new do |initial_arg|
  puts initial_arg # => 2. Begin Worker
  puts parent.transfer("3. Resume Parent") # => 4. Resume Worker
  "5. Exit Worker"
end

puts "1. Start"
puts worker.transfer("2. Begin Worker")
# => "3. Resume Parent"
puts worker.transfer("4. Resume Worker")
# => "5. Exit Worker"
puts "6. Exit Script"

But it behaves differently when inside a surrounding, non-main Fiber:

manager = Fiber.new do |manager_init|
  puts manager_init # => 1. Start
  parent = Fiber.current
  worker = Fiber.new do |initial_arg|
    puts initial_arg # => 2. Begin Worker
    puts parent.transfer("3. Resume Parent")
    "5. Exit Worker"
  end

  puts worker.transfer("2. Begin Worker")
  # => "3. Resume Parent"
  puts worker.transfer("4. Resume Worker")
  # => "5. Exit Worker"

  "6. Exit Script" # This isn't printed
end

puts manager.transfer("1. Start")

# 1. Start
# 2. Begin Worker
# 3. Resume Parent
# 4. Resume Worker
# 5. Exit Worker
# 
# ... (6 doesn't print here)

Maybe the first example is a special case, since it's the main Fiber. I'd be happy to add a comment about that to the Fiber docs if that'd be helpful!

[ioquatix (Samuel Williams)]

In think clarifying the documentation is a good idea.

And .transfer does return to the caller, as long as the caller is the main Fiber, for example, this prints out a sequence of messages:

I think that's just intuitively correct based on the semantics.

Actually, one could argue that a fiber that was transferred to, that exits, without an explicit transfer, should be considered exiting the thread completely and in your case terminating the program. In this case, Ruby is being a bit generous with "transferring back to the main fiber" IMHO. Transfer should be seen as a one way operation.