Feature #19326: Please add a better API for passing a Proc to a Ractor

• Status: Assigned
• Assignee: ko1 (Koichi Sasada)
• Target version: -

[sdwolfz (Codruț Gușoi)]

Example 1:

class Worker
  def initialize(&block)
    @block = block
  end

  def run
    Ractor.new(@block, &:call)
  end
end

worker = Worker.new { 1 }
puts worker.run.take

Errors with:

<internal:ractor>:271:in `new': allocator undefined for Proc (TypeError)
        from scripts/run.rb:9:in `run'
        from scripts/run.rb:14:in `<main>'

Example 2:

class Worker
  def initialize(&block)
    @block = Ractor.make_shareable(block)
  end

  def run
    Ractor.new(@block, &:call)
  end
end

worker = Worker.new { 1 }
puts worker.run.take

Errors with:

<internal:ractor>:820:in `make_shareable': Proc's self is not shareable: #<Proc:0x00007f00394c38b8 scripts/run.rb:13> (Ractor::IsolationError)
        from scripts/run.rb:5:in `initialize'
        from scripts/run.rb:13:in `new'
        from scripts/run.rb:13:in `<main>'

Example 3:

class Worker
  def initialize(&block)
    @block = Ractor.make_shareable(block)
  end

  def run
    Ractor.new(@block, &:call)
  end
end

worker = Ractor.current.instance_eval { Worker.new { 1 } }
puts worker.run.take

Works, but having Ractor.current.instance_eval as a wrapper around the block is not ideal, as Ractor is supposed to be only an implementation detail in Worker.

I know about https://bugs.ruby-lang.org/issues/18243 and the discussion around proc.bind(nil). That would actually be ideal, as for the purposes if why I want this functionality I don't care what self is in a block, and the less it has access to the better.

The general idea of Worker is to have a Ractor be able to lazily execute an arbitrary proc. And all the bindings it would need would be passed explicitly, either through args in the constructor or through send/receive, so self would really not matter.

The benefit: this would make it so concurrent code can be more easily be implemented with Ractors as currently you can execute an arbitrary proc by passing it to a Thread (but you don't get the nice data isolation).

[sdwolfz (Codruț Gușoi)]

• Description updated (diff)

[luke-gru (Luke Gruber)]

If you want Ractor to be an implementation detail of Worker you could do:

class Worker
  def initialize(&block)
    @block = block
  end

  def run
    block = @block
    true.instance_eval { Ractor.new(&block) }
  end
end

worker = Worker.new { 1 }
puts worker.run.take

Just an idea :)

It is strange that self is checked even when Ractor.new is given a block not yet marked shareable, because the given block's self is changed to the current ractor during execution. Ractor.make_shareable(), however, makes sense to check self.

[sdwolfz (Codruț Gușoi)]

OK, I had absolutely no idea you could do that. Thanks for the suggestion! My impressions was instance_eval could only be used around where a block is written.

Can we document this behaviour somewhere? Maybe in https://docs.ruby-lang.org/en/3.2/Ractor.html right after the Shareable and unshareable objects section. I suspect many more people will hit the same wall as I had and would not have an easy time figuring it out.

[Eregon (Benoit Daloze)]

I'm not sure this behavior is intended, cc @ko1 (Koichi Sasada).
I thought Ractor.new would only accept literal blocks.
The problem is the usual "does not respect the intent of the writer of the block, which expected a specific self".
Also it only works if you are fine to create a Ractor per block, which seems a big overhead.

Maybe Proc#isolate(new_self) or so should be exposed.
OTOH Ractor is incompatible with most gems out there, so seems of limited usefulness.
For true parallelism compatible with existing Ruby code, use threads on TruffleRuby/JRuby.

[sdwolfz (Codruț Gușoi)]

I thought Ractor.new would only accept literal blocks.

I hope not, passing an arbitrary block helps with building functionality on top of Ractors (that I want to do).

The problem is the usual "does not respect the intent of the writer of the block, which expected a specific self".

My "Worker" class will have documentation stating that self is replaced and all dependencies need to be passed through the constructor so that's not going to be an issue.

Also it only works if you are fine to create a Ractor per block, which seems a big overhead.

I could have a "Ractor pool" basically limiting the number of Ractors that can run at the same time. Still, the overhead could be improved with future Ruby releases (just like it is for Erlang/Elixir). For all practical purposes so far the overhead is negligible compared to the work done in the blocks.

Maybe Proc#isolate(new_self) or so should be exposed.

This would be a nice functionality to have, indeed.

OTOH Ractor is incompatible with most gems out there, so seems of limited usefulness.

It will become better when Ractor is stabilised, right now I have all I need working.

For true parallelism compatible with existing Ruby code, use threads on TruffleRuby/JRuby.

I'd rather use the real Ruby and stay away from Oracle.

All that being said, I'm really interested to see how this functionality evolves.

[sdwolfz (Codruț Gușoi)]

New example:

class Worker
  def initialize(&block)
    @block = block
  end

  def start
    block = Ractor.make_shareable(@block)

    @ractor = Ractor.new(block) do |callable|
      message = receive

      callable.call(message)
    end

    self
  end

  def work(*args)
    @ractor.send(args)

    @ractor.take
  end
end

worker = Ractor.current.instance_eval do
  Worker.new { |args| puts args.inspect }
end

worker.start.work(1, 2, 3)

Say I don't want to just pass the bock as is to the Ractor, but I want to do some setup first within the Ractor itself. In that case I'm back to needing both make_shareable and the instance_eval around the literal block.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Assigned
• Assignee set to ko1 (Koichi Sasada)

[luke-gru (Luke Gruber)]

Your new example, I think, is not possible with the current API.

However, there is a bug in Ruby that I just found that allows it.

class Worker
  def start(&blk)
    blk = blk.curry # bug in ruby allows sharing of non-shareable proc
    Ractor.make_shareable(blk)
    @ractor = Ractor.new(blk) do |b|
      message =  receive
      b.call(message)
    end

    self
  end

  def work(msg)
    @ractor.send(msg)

    @ractor.take
  end
end

worker = Worker.new
worker.start { |args| p args }

worker.work('msg')

This is a bug because in debug versions of ruby this crashes. I opened a ticket for it: https://bugs.ruby-lang.org/issues/19374

[forthoney (Seong-Heon Jung)]

Are there any updates on this issue?