Feature #21550: Ractor.shareable_proc/shareable_lambda to make sharable Proc object

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: 3.5

[ko1 (Koichi Sasada)]

Let's introduce a way to make a sharable Proc.

• Ractor.shareable_proc(self: nil, &block) makes proc.
• Ractor.shareable_lambda(self: nil, &block) makes lambda.

See also: https://bugs.ruby-lang.org/issues/21039

Background

Motivation

Being able to create a shareable Proc is important for Ractors. For example, we often want to send a task to another Ractor:

worker = Ractor.new do
  while task = Ractor.receive
    task.call(...)
  end
end

task = (shareable_proc)
worker << task

task = (shareable_proc)
worker << task

task = (shareable_proc)
worker << task

There are various ways to represent a task, but using a Proc is straightforward.

However, to make a Proc shareable today, self must also be shareable, which leads to patterns like:

  nil.instance_eval{ Proc.new{ ... } }

This is noisy and cryptic. We propose dedicated methods to create shareable Proc objects directly.

Specification

• Ractor.shareable_proc(self: nil, &block) makes a proc.
• Ractor.shareable_lambda(self: nil, &block) makes a lambda.

Both methods create the Proc/lambda with the given self and make the resulting object shareable.

(changed) Accessing outer variables are not allowed. An error is raised at the creation.

More about outer-variable handling are discussed below.

In other words, from the perspective of a shareable Proc, captured outer locals are read‑only constants.

This proposal does not change the semantics of Ractor.make_shareable() itself.

Discussion about outer local variables

[Feature #21039] discusses how captured variables should be handled.
I propose two options.

0. Disallow accessing to the outer-variables

It is simple and no confusion.

1. No problem to change the outer-variable semantics

@Eregon (Benoit Daloze) noted that the current behavior of Ractor.make_shareable(proc_obj) can surprise users. While that is understandable, Ruby already has similar surprises.

For instance:

RSpec.describe 'foo' do
  p self #=> RSpec::ExampleGroups::Foo
end

Here, self is implicitly replaced, likely via instance_exec.
This can be surprising if one does not know self can change, yet it is accepted in Ruby.
We view the current situation as a similar kind of surprise.

2. Enforce a strict rule for non‑lexical usage

The difficulty is that it is hard to know which block will become shareable unless it is lexically usage.

# (1) On this code, it is clear that the block will be shareable block:

a = 42
Ractor.shareable_proc{
  p a
}

# (2) On this code, it is not clear that the block becomes sharable or not
get path do
  p a
end

# (3) On this code, it has no problem because
get '/hello' do
  "world"
end

The idea is to allow accessing captured outer variables only for lexically explicit uses of Ractor.shareable_proc as in (1), and to raise an error for non‑lexical cases as in (2).
So the example (3) is allowed if the block becomes sharable or not.

The strict rule is same as Ractor.new block rule.

3. Adding new rules

(quoted from https://bugs.ruby-lang.org/issues/21550#note-7)

Returning to the issue: we want a way to express that, within a block, an outer variable is shadowed while preserving its current value.

We already have syntax to shadow an outer variable using |i; a|, where a is shadowed in the block and initialized to nil (just like a normal local variable).

a = 42
pr = proc{|;a| p a}
a = 43
pr.call #=> nil

What if we instead initialized the shadowed variable to the outer variable's current value?

a = 42
pr = proc{|;a| p a}
a = 43
pr.call #=> 42

For example, we can write the port example like that:

port = Ractor::Port.new
Ractor.new do |;port|
  port << ...
end

and it is better (shorter).

Maybe only few people know this spec and I checked that there are few lines in rubygems (78 cases in 3M files)(*1).
So I think there is a few compatibility impact.

[ko1 (Koichi Sasada)]

• Description updated (diff)

[ko1 (Koichi Sasada)]

• Description updated (diff)

[ko1 (Koichi Sasada)]

• Related to Feature #21039: Ractor.make_shareable breaks block semantics (seeing updated captured variables) of existing blocks added

[ko1 (Koichi Sasada)]

• Description updated (diff)

[matz (Yukihiro Matsumoto)]

I understand the concern for future confusion, but it's a trade-off. I'd accept confusion here (option 1) to avoid complex semantics and implementation.

Matz.

[Eregon (Benoit Daloze)]

Summary: I think option 2 is great it's both flexible, clear and safe, and we should implement it.
It addresses the concerns in #21039.
It should be easy to implement, because if given a Proc it's the same semantics as Ractor.new(proc_object), and when given a literal block it's the desired more flexible semantics which anyway both options want.

What follow is some more thoughts, which might lead to some improvements later but anyway I believe it's fine to implement option 2 as-is.

---

I wonder if the ability to capture outer local variables for shareable procs is even needed at all.
Because that only works if the value of the captured variables is shareable, which seems not so likely (only numbers/true/false/nil/frozen-string-literals/modules/regexps are frozen/shareable without an explicit freeze/make_shareable call).
I would like to see some concrete examples of blocks to be made shareable which capture outer local variables, and how they handle the case of captured variable values not already being deeply frozen (cc @tenderlovemaking (Aaron Patterson) you said there were some in Rails IIRC).

Related to that, maybe it would be useful for

o = Object.new
Ractor.shareable_proc {
  o
}

to call Ractor.make_shareable(o)?
Or maybe Ractor.make_shareable(o, copy: true) on Ractor.shareable_proc(copy: true) { ... }?
Though not sure if reasonable or too surprising. It might be surprising for cases where it freezes many objects, e.g. if o is Foo.new([]) the Array also gets frozen (which might break things).
I think that case makes it clear this might maybe be acceptable with a literal block, but would never be acceptable with a Proc object (it would freeze things far away and be hard to debug).

In some way we can see a block as an object with an extra @ivar being the "captured environment" which contains the captured variables.
From that POV, making the Proc shareable then would rather naturally make that "captured environment" and objects inside shareable as well.

---

I find it interesting that we all seem to agree on Assigning to outer local variables from within the shareable Proc is not allowed (error at creation) but not on Assigning to outer local variables from outside the shareable Proc is not allowed through an exception when trying to create a shareable Proc in such a case.
IMO they are just both sides of the same issue.
But I understand the second one is more tricky because e.g. it's impossible to detect assignments within eval at (file) parse time (IMO so rare of an edge case to not worry about that too much).
I also agree the second one is more rarely used, though that doesn't mean it doesn't matter.

---

With option 1 it would break existing blocks, so I am against it, e.g.:

visits = 0
get '/' do # assuming this block is not made shareable (because that is forbidden, see `Assigning to outer local variables from within the shareable Proc is not allowed (error at creation)`)
  visits += 1
  "Hello"
end
get '/visits' do # if this block is made shareable and called, it will behave incorrectly and use a snapshot of `visits`
  visits.to_s
end

Concretely, assuming the second block is made shareable at load time, if the block is used only in non-main Ractors but not in main Ractors:

• if the request hits the main Ractor it would be correct (the number of visits of /)
• if it hits a non-main Ractor it would be incorrect (0).

That's pretty bad for obvious reasons.
If the shared block is instead used by all Ractors, including the main one then it would always be incorrect (0). Still clearly breaking the program/intended semantics.

So standard Ruby code, specifically blocks (and their intended logic) would be broken "just because a Proc has been made Ractor-shareable".

This is worse than just instance_exec, because a block called with instance_exec is typically always called with the same kind of receiver.
And the worse case with instance_exec is a NoMethodError or calling the wrong method (very unlikely).
The worse case with option 1 is reading a stale/outdated/inconsistent value, very much like a stale read which is usually a race condition/multithreading problem, but here Ractor would actually introduce this issue even though Ractor is meant to prevent such issues.

[ko1 (Koichi Sasada)]

Eregon (Benoit Daloze) wrote in #note-6:

I wonder if the ability to capture outer local variables for shareable procs is even needed at all.
Because that only works if the value of the captured variables is shareable, which seems not so likely (only numbers/true/false/nil/frozen-string-literals/modules/regexps are frozen/shareable without an explicit freeze/make_shareable call).
I would like to see some concrete examples of blocks to be made shareable which capture outer local variables, and how they handle the case of captured variable values not already being deeply frozen (cc @tenderlovemaking (Aaron Patterson) you said there were some in Rails IIRC).

I have two examples.

One is define_method.

def define_foo suffix, ivar_name # should be in Symbol
  define_method "foo_#{mid}" do
    instance_variable_get ivar_name
  end
end

This method define a new method by using a outer local variable ivar_name.

For example, this pattern is used:

# lib/delegate.rb
def Delegator.delegating_block(mid) # :nodoc:
  lambda do |*args, &block|
    target = self.__getobj__
    target.__send__(mid, *args, &block)
  end.ruby2_keywords
end

# lib/bundler/errors.rb
    def self.status_code(code)
      define_method(:status_code) { code }


# repl_type_completor/test/repl_type_completor/test_repl_type_completor.rb
    def with_failing_method(klass, method_name, message)
      ...
      # message should be marked as sharable
      klass.define_method(method_name) do |*, **|
        raise Exception.new(message)
      end

# debug/test/console/config_fork_test.rb
    ['fork', 'Process.fork', 'Kernel.fork'].each{|fork_method|
      c = Class.new ConsoleTestCase do
        ...
        # fork_method shouldbe marked as sharable
        define_method :fork_method do
          fork_method
        end
      end

The first motivation is how to handle define_method.

Another example to make a task object which should be run in another Ractor.

result = Ractor::Port.new
Ractor.shareable_proc do
  result << task()
end

p result.receive

I have already written the following pattern many times:

port = Ractor::Port.new
Ractor.new port do |port|
  port << ...
end

and it is somewhat cumbersome to write.

---

At least we have no objection to introduce Ractor.shareable_proc(&bl) if bl doesn't have references to the outer variable.
I'll merge it.

---

I have another idea.

Returning to the issue: we want a way to express that, within a block, an outer variable is shadowed while preserving its current value.

We already have syntax to shadow an outer variable using |i; a|, where a is shadowed in the block and initialized to nil (just like a normal local variable).

a = 42
pr = proc{|;a| p a}
a = 43
pr.call #=> nil

What if we instead initialized the shadowed variable to the outer variable's current value?

a = 42
pr = proc{|;a| p a}
a = 43
pr.call #=> 42

For example, we can write the port example like that:

port = Ractor::Port.new
Ractor.new do |;port|
  port << ...
end

and it is better (shorter).

Maybe only few people know this spec and I checked that there are few lines in rubygems (78 cases in 3M files)(*1).
So I think there is a few compatibility impact.

*1:

# {name: foo}, 3] means shadowed name "foo" is used 3 times.

[[[:FILES, 3_068_818], [:FAILED_PARSE, 14_928]],
 [[:lvar, 78],
  [{name: :glark}, 6],
  [[:FAILED, SystemStackError], 5],
  [{name: :bl}, 4],
  [{name: :a}, 4],
  [{name: :options}, 3],
  [{name: :name}, 3],
  [{name: :bytes}, 2],
  [{name: :extra}, 2],
  [{name: :shape}, 2],
  [{name: :b}, 2],
  [{name: :x}, 2],
  [{name: :new_root}, 2],
  [{name: :bl2}, 2],
  [{name: :ifc_spc}, 2],
  [{name: :m}, 2],
  [{name: :foo}, 2],
  [{name: :key}, 2],
  [{name: :md}, 2],
  [{name: :d}, 2],
  [{name: :c}, 2],
  [{name: :out}, 2],
  [{name: :bar}, 2],
  [{name: :req}, 1],
  [{name: :resp}, 1],
  [{name: :spec}, 1],
  [{name: :world}, 1],
  [{name: :u}, 1],
  [{name: :onto}, 1],
  [{name: :rbname}, 1],
  [{name: :r}, 1],
  [{name: :h}, 1],
  [{name: :bug9605}, 1],
  [{name: :expected}, 1],
  [{name: :result}, 1],
  [{name: :services}, 1],
  [{name: :step}, 1],
  [{name: :obj}, 1],
  [{name: :ancestor}, 1],
  [{name: :count}, 1],
  [{name: :path}, 1],
  [{name: :nav}, 1],
  [{name: :safe_position_in_input}, 1],
  [{name: :safe_count}, 1],
  [{name: :v}, 1],
  [{name: :k}, 1],
  [{name: :indent}, 1],
  [{name: :y}, 1],
  [{name: :z_diff}, 1]]]

[ko1 (Koichi Sasada)]

• Description updated (diff)

[ko1 (Koichi Sasada)]

I made a patch to disallow accessing to any outer variables and I found that I can't write ostruct case:

# ostruct-0.6.3/lib/ostruct.rb

  def new_ostruct_member!(name) # :nodoc:
    unless @table.key?(name) || is_method_protected!(name)
      if defined?(::Ractor.shareable_proc)
        getter_proc = Ractor.shareable_proc { @table[name] }
        setter_proc = Ractor.shareable_proc {|x| @table[name] = x}
      elsif defined?(::Ractor)
        getter_proc = nil.instance_eval{ Proc.new { @table[name] } }
        setter_proc = nil.instance_eval{ Proc.new {|x| @table[name] = x} }
        ::Ractor.make_shareable(getter_proc)
        ::Ractor.make_shareable(setter_proc)
      else
        getter_proc = Proc.new { @table[name] }
        setter_proc = Proc.new {|x| @table[name] = x}
      end
      define_singleton_method!(name, &getter_proc)
      define_singleton_method!("#{name}=", &setter_proc)
    end
  end

So I think it is needed to allow the way to access outer variables.
Option 2 (allow accesses only on lexical blocks) is rejected by Matz, so I want to introduce 1 or 3.
1 is better for me...

[tenderlovemaking (Aaron Patterson)]

ko1 (Koichi Sasada) wrote in #note-9:

I made a patch to disallow accessing to any outer variables and I found that I can't write ostruct case:

# ostruct-0.6.3/lib/ostruct.rb

  def new_ostruct_member!(name) # :nodoc:
    unless @table.key?(name) || is_method_protected!(name)
      if defined?(::Ractor.shareable_proc)
        getter_proc = Ractor.shareable_proc { @table[name] }
        setter_proc = Ractor.shareable_proc {|x| @table[name] = x}
      elsif defined?(::Ractor)
        getter_proc = nil.instance_eval{ Proc.new { @table[name] } }
        setter_proc = nil.instance_eval{ Proc.new {|x| @table[name] = x} }
        ::Ractor.make_shareable(getter_proc)
        ::Ractor.make_shareable(setter_proc)
      else
        getter_proc = Proc.new { @table[name] }
        setter_proc = Proc.new {|x| @table[name] = x}
      end
      define_singleton_method!(name, &getter_proc)
      define_singleton_method!("#{name}=", &setter_proc)
    end
  end

So I think it is needed to allow the way to access outer variables.
Option 2 (allow accesses only on lexical blocks) is rejected by Matz, so I want to introduce 1 or 3.
1 is better for me...

I think 3 is a good idea, but I think we could introduce it in later versions of Ruby and slowly migrate code (if the syntax change is acceptable). Option 1 lets us more easily port existing code without syntax changes, so I like it for now.

[Eregon (Benoit Daloze)]

ko1 (Koichi Sasada) wrote in #note-9:

Option 2 (allow accesses only on lexical blocks) is rejected by Matz, so I want to introduce 1 or 3.

Where did matz say that? In https://bugs.ruby-lang.org/issues/21550#note-5 he said:

I'd accept confusion here (option 1) to avoid complex semantics and implementation.

I'd accept sounds like a preference, not rejection.
As I clarified in my comment after that, I see no concern about complex implementation, it's the same effort anyway:

It should be easy to implement, because if given a Proc it's the same semantics as Ractor.new(proc_object), and when given a literal block it's the desired more flexible semantics which anyway both options want.

And regarding complex semantics (due to different behavior with literal vs non-literal block) it's necessary to avoid breaking block semantics, which are way more fundamental than Ractor.shareable_proc semantics.
Kernel#lambda already had such semantics BTW (no effect if not a literal block), and in fact for similar reasons and concerns (don't break the author of the block's intent).
So there is precedent for doing exactly this kind of switching on literal block vs Proc.

Option 2 seems the obvious and correct solution for the ostruct case.

Though it seems that define_method is a frequent issue that keeps coming up with Ractor, and that's quite a specific case where we know the Proc self will be changed anyway.
So I think define_method(name) { @table[name] } / define_singleton_method(name) { @table[name] } should just work, with and without Ractors.
That would mean that define_method would automatically shallow-copy the environment, Ractor or not, for consistency. That's a semantic change but it seems very compatible.
And define_method knows self will be the instance so no need to check if self is shareable.
And the defined method would automatically be available to Ractors, as long as the captured variables values are shareable.
If they are not, then the method can't be called on a Ractor, that's impossible anyway, same error as currently.

That would mean define_method would just work with Ractor, vs needing pretty messy code as in the ostruct example. "if Ractor" code should be avoided as much as possible.

If we are too concerned about changing define_method semantics we could instead make that behavior opt-in e.g. via a keyword argument to define_method like define_method(name, ractor/shareable/make_shareable: true) { @table[name] } (which can work on older Rubies by passing **kwargs and making it empty on Ruby < 3.5).

EDIT: See https://bugs.ruby-lang.org/issues/21550#note-13, that resolves any compatibility problem transparently + make define_method work with Ractor automatically for almost all cases.

[Eregon (Benoit Daloze)]

Option 3 doesn't seem good because it would break the block if the block is run on Ruby 3.4 and before, as the value of a would be nil.
One would need to have 2 copies of the block which is clearly inconvenient.

I think Ractor should be able to use captured variables, because this is one of the most elegant ways to pass data/input in Ruby.
But it should be safe, and that means not breaking normal Ruby block semantics (at least for blocks which don't explicitly opt-in to the environment copy behavior).

So I think Option 4 would still be the best, and that's the option proposed in https://bugs.ruby-lang.org/issues/21039#note-21:

• Ractor.shareable_proc is the same for literal and non-literal blocks
• Ractor.shareable_proc raises if it captures a variable which is reassigned inside or after the block/Proc given to Ractor.shareable_proc.

Notice we already agree on "Ractor.shareable_proc raises if it captures a variable which is reassigned inside the block/Proc given to Ractor.shareable_proc".
So it's just adding or after, which makes it fully safe and compatible with regular block semantics.
I don't think that check is very difficult to implement, in fact I can help implementing it.
Ruby code to make it clear:

# OK because does not change the semantics of b
def example
  a = 1
  a += 2
  b = proc { a }
  Ractor.shareable_proc(&b)
end

# OK because does not change the semantics of the block
def example
  a = 1
  a += 2
  Ractor.shareable_proc { a }
end

# error (that everyone seems to agree on)
def example
  a = 1
  b = proc { a = 2 }
  Ractor.shareable_proc(&b)
end

# error (that everyone seems to agree on)
def example
  a = 1
  Ractor.shareable_proc { a = 2 }
end

# error (the case we discussing about): the code clearly assumes it can reassigns `a` but the `shareable_proc` would not respect it, i.e. `shareable_proc` would break Ruby block semantics
# Also note the Ractor.shareable_proc call might be far away from the block, so one can't tell when looking at the block that it would be broken by `shareable_proc` (if no error for this case)
def example
  a = 1
  b = proc { a }
  Ractor.shareable_proc(&b)
  a = 2
end

# I think should be error too, semantics are ill-defined in such a case, the code clearly assumes it can reassigns `a` but the `shareable_proc` would not respect it
def example
  a = 1
  Ractor.shareable_proc { a }
  a = 2
end

That check can be more strict for convenience (e.g. raise if the assignments are not trivially all before the block/Proc),
I think that's fine because it's fairly rare to reassign a variable after a block captures it, so won't be a practical limitation anyway.
But still, such cases should still behave correctly according to block semantics, hence should be forbidden for Ractor.shareable_proc.
Re eval and binding it's so rare and such a corner case in combination with reassigning a variable after a block captures it that I think it is acceptable to expose that Ractor.shareable_proc copies the environment for those extremely rare cases.
BTW, Proc#binding is already not supported for a shareable_proc:

$ ruby -e 'nil.instance_exec { a = 1; b = proc { a }; b2 = Ractor.make_shareable(b); p b2.binding }'
-e:1:in `binding': Can't create Binding from isolated Proc (ArgumentError)

So binding/eval is in general already not fully respected with Ractor anyway.

Some edge cases for clarity:

# OK, we cannot detect it, extreme corner case unlikely to appear in any real code. The shareable_proc will capture 1, b will capture 2.
def example
  a = 0
  b = proc { a }
  p = Ractor.shareable_proc(&b)
  eval("a = 2") # or binding.local_variable_set(:a, 2), or b.binding.local_variable_set(:a, 2)
  [b, p]
end

# error, `a` is reassigned after the block
def example
  a = 0
  while condition
    b = proc { a }
    p = Ractor.shareable_proc(&b)
    a += 1
  end
end

# error, `a` might be reassigned (if condition is true twice or more, but we have to analyze statically so be conservative)
def example
  while condition
    a = rand
    b = proc { a }
    p = Ractor.shareable_proc(&b)
  end
end

As said before, I'm OK with option 2, but it's less flexible than option 4.
We could also have a mix and allow everything for literal block, and option 4 for Proc.
I remain strongly against option 1, I think it is a language design mistake we won't be able to fix later.

And for define_method I think we should have something more convenient than Ractor.shareable_proc (see previous comment).

[Eregon (Benoit Daloze)]

Regarding define_method, it's as-if it would try Ractor.shareable_proc with option 4 semantics automatically on the block/Proc, and if that succeeds use that, if it fails then make the method not callable from a Ractor and use block/Proc as-is.
So that would mean no compatibility issue if a captured variable is reassigned inside or after the block, and for all other cases it just works with Ractor.

[ko1 (Koichi Sasada)]

@Eregon (Benoit Daloze) let's me clarify your proposal.

• Option 4 is your proposal on https://bugs.ruby-lang.org/issues/21039#note-21, which prohibit any writing to the captured outer variables from inside/outside of the block
  • This proposal is clearly rejected by Matz https://bugs.ruby-lang.org/issues/21039#note-28
• About define_method

  • That would mean that define_method would automatically shallow-copy the environment, Ractor or not, for consistency. That's a semantic change but it seems very compatible.

  • I think it is not acceptable because it will break compatibility if the block uses outer local variables as a storage, like that:

    def define_counters suffix, cnt = 0
      define_method("get_#{suffix}){ cnt     }
      define_method("inc_#{suffix}){ cnt += 1}
    end
    

    I haven't count such cases in real codebase yet.

[ko1 (Koichi Sasada)]

I confirmed with Matz that

• option 2 is too complex and therefore rejected.
• option 3 is also rejected because |;n| represents shadowing, not capturing.
• option 1 is preferable: not the best, but the better one among these.

(I met with Matz today and discussed)

So I will merge option 1 soon.
Please reopen if there are further points.

[Eregon (Benoit Daloze)]

ko1 (Koichi Sasada) wrote in #note-14:

@Eregon (Benoit Daloze) let's me clarify your proposal.

• Option 4 is your proposal on https://bugs.ruby-lang.org/issues/21039#note-21, which prohibit any writing to the captured outer variables from inside/outside of the block

I think there is a misunderstanding there: Option 4 does not prohibit writing to captured variables, it never proposed that. Ruby code can write to all local variables, always.
Maybe the text in that comment is confusing (e.g. "Disallow writes" is not to literally disallow writing, it's to prevent making a shareable proc in such a context).
Please read https://bugs.ruby-lang.org/issues/21550#note-12 and ignore that older comment, it should be very clear what is proposed with all the examples.
It prevents making a shareable_proc for a block which uses captured variables which are reassigned after the block, as I showed in the examples above:

def example
  a = 1
  b = proc { a }
  Ractor.shareable_proc(&b) # should be Ractor::IsolationError: cannot isolate a block because it accesses outer variables (a) which are reassigned inside or after the block
  a = 2
end

All options already prevent making a shareable_proc for a block which uses captured variables which are reassigned inside the block.

# error (that everyone seems to agree on)
def example
  a = 1
  b = proc { a = 2 }
  Ractor.shareable_proc(&b)
end

So option 4 does the same but also prevents making a shareable proc when reassigned after.

• This proposal is clearly rejected by Matz https://bugs.ruby-lang.org/issues/21039#note-28

I think the proposal was misunderstood, I tried to clarify in https://bugs.ruby-lang.org/issues/21039#note-33 as well.
"prohibiting outer assignment to local variables when a proc is made sharable." was never proposed (because that's clearly too difficult and the wrong place to prevent/prohibit).

• About define_method

  • That would mean that define_method would automatically shallow-copy the environment, Ractor or not, for consistency. That's a semantic change but it seems very compatible.

  • I think it is not acceptable because it will break compatibility if the block uses outer local variables as a storage, like that:

My last comment explains how to fix this.
Since it's not clear I will make a separate proposal which does not rely on any previous comment to minimize confusion.

[Eregon (Benoit Daloze)]

• Subject changed from Ractor.sharable_proc/sharable_lambda to make sharable Proc object to Ractor.shareable_proc/shareable_lambda to make sharable Proc object
• Description updated (diff)

[Eregon (Benoit Daloze)]

ko1 (Koichi Sasada) wrote in #note-15:

So I will merge option 1 soon.
Please reopen if there are further points.

Please wait, I believe you and matz did not understand my proposal.
I will make a clear proposal ASAP.

[ko1 (Koichi Sasada)]

I think there is a misunderstanding there: Option 4 does not prohibit writing to captured variables, it never proposed that. Ruby code can write to all local variables, always.

Ok I misunderstood.

def sh(bl)
  Ractor.sharable_proc(&bl)
  #=> raise an error because threre is `a = 2` line.
end

a = 1
shpr = sh proc{ a }
a = 2

Is it correct?

To make it we need to track local assignment outside of the block.
Is that feasible...?

[Eregon (Benoit Daloze)]

Yes, that's the idea.
Yes it's feasible via static analysis (e.g. in compile.c).
It would be somewhat conservative but I think that's good enough, I'll give more details in my proposal.
I think a possible simple way is to use "if a captured variable is assigned more than once, Ractor.sharable_proc on a block using that captured variable gives Ractor::IsolationError".

[Eregon (Benoit Daloze)]

I wrote my detailed proposal in #21557.

[Eregon (Benoit Daloze)]

• Related to Feature #21557: Ractor.shareable_proc to make sharable Proc objects, safely and flexibly added

[ko1 (Koichi Sasada)]

• Status changed from Open to Closed

Applied in changeset git|55b1ba3bf276ba82173bd961fb8e0f08bf4182a6.

---

Ractor.shareable_proc

call-seq:
Ractor.sharable_proc(self: nil){} -> sharable proc

It returns shareable Proc object. The Proc object is
shareable and the self in a block will be replaced with
the value passed via self: keyword.

In a shareable Proc, the outer variables should

• (1) refer shareable objects
• (2) be not be overwritten

  a = 42
  Ractor.shareable_proc{ p a }
  #=> OK

  b = 43
  Ractor.shareable_proc{ p b; b = 44 }
  #=> Ractor::IsolationError because 'b' is reassigned in the block.

  c = 44
  Ractor.shareable_proc{ p c }
  #=> Ractor::IsolationError because 'c' will be reassigned outside of the block.
  c = 45

  d = 45
  d = 46 if cond
  Ractor.shareable_proc{ p d }
  #=> Ractor::IsolationError because 'd' was reassigned outside of the block.

The last d's case can be relaxed in a future version.

The above check will be done in a static analysis at compile time,
so the reflection feature such as Binding#local_varaible_set
can not be detected.

  e = 42
  shpr = Ractor.shareable_proc{ p e } #=> OK
  binding.local_variable_set(:e, 43)
  shpr.call #=> 42 (returns captured timing value)

Ractor.sharaeble_lambda is also introduced.
[Feature #21550]
[Feature #21557]