Project

General

Profile

Actions

Feature #17284

closed

Shareable Proc

Added by ko1 (Koichi Sasada) over 3 years ago. Updated over 3 years ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-core:100534]

Description

For some reasons, we need to provide a way to make sharable Proc between ractors.

  • (1) A block for the Ractor.new.
  • (2) Send a proc between ractors.
  • (3) A block for global callback methods: define_method ([Bug #17159]), TracePoint, ...

For (1), we use Proc#isolate (isolate is temporary name here) which prohibit to access outer variables.

a = 1
Proc.new{
  p a 
}.isolate # => can not isolate a Proc because it accesses outer variables (a).
          # error on `isolate` method call

There are no states to share, so it is okay.

For (2), Proc#isolate is one option because we can send parameters with an argument call.
But it should be a bit long.

i, j, k = nil

pr = Proc.new do |i, j, k|
  p i, j, k
end.isolate

r = Ractor.new do |task, param|
  task.call(*param)
end

r.send([pr, [i, j, k]])

For (3), maybe we need to make more flexible Proc which can read outer block parameter on that snapshot (discussed in #17159).

Now, I named it with freeze, because it seems frozen Proc.

a = 1

# try to read, and returns old value (snapshot at `freeze`)
pr = Proc.new{
  p a #=> 1
}
pr = pr.freeze
pr.call

a = 2

pr.call #=> 1


# try to write, and it is not allowed
pr2 = Proc.new{
  a = 1
}
pr2 = pr.freeze
#=> can not freeze a Proc because it accesses outer variables (a). (ArgumentError)

To share the "frozen" Proc between ractors, outer values should be (deep) frozen. It means readable values (in above case, a) should be shareable.
Now we named it Proc#shareable!

a = [1, [2, 3]]
pr = Proc.new{
  p a.frozen? #=> true
}.shareable!

a[0] = 0 #=> frozen error

This ticket has three different variant of mutability and shareability for Proc.

outer lvar shareable freeze/making shareable other objects
a. isolate N/A Yes No
b. freeze allow to read No No
c. shareable! allow to read Yes Yes

I want to introduce functionality of shareable!, but not sure the Ruby-level API.

I think (b) freeze for this semantics is good name because it only allows to read-only local variables.
However, it is not enough to make a sharable Proc because read objects from the Proc should be also sharable.

Making freeze with (c) shareable! functionality is one idea, but I think freeze should not deep-freezing because it is very surprising that read objects become the sharable (== frozen) for usual Ruby users.
Maybe Ractor.make_sharable(pr) makes pr sharable is no surprise because it is good declaration the pr should be shareable, even if the read objects from pr become shareable (== frozen).

Removing (a) isolate and using (c) shareable! at Ractor.new(&b) is one idea, but I think it is surprising that they can access outer local variables, but the they can not access newly assigned variables as usual blocks.

a = 1
Ractor.new do
  p a # only 1
end

a = 2

(a) isolate does not have such issue because all outer lvars accesses are not allowed == easy to understand, easy to debug.

In practice, accessing outer variables with multi-ractor program is very useful because we need to declare same local variables if we want to access them from different ractors.

The following example is from [Feature #17261]:

tv1 = Thread::TVar.new(0)
tv2 = Thread::TVar.new(0)

r1 = Ractor.new tv1, tv2 do |tv1, tv2|    # <-- here
  loop do
    Thread.atomically do
      v1, v2 = tv1.value, tv2.value
      raise if v1 != v2
    end
  end
end

With (c) shareable! semantics, it is easier to write:

tv1 = Thread::TVar.new(0)
tv2 = Thread::TVar.new(0)

r1 = Ractor.new do
  loop do
    Thread.atomically do
      v1, v2 = tv1.value, tv2.value
      raise if v1 != v2
    end
  end
end

Above example is also enable to make more simple:

i, j, k = nil

pr = Proc.new do
  p i, j, k
end

r = Ractor.new do |task|
  task.call
end

r.send(pr)

However, using this semantics (shareable!) can freeze extra-variables in accidents:

a = [1, 2, 3]

Ractor.new do
  do_something if a.length > 0
end

a << 4 # raises FrozenError

It is clear that there is a syntax or method to apply shareable! functionality.

a = [1, 2, 3]
Ractor.new &(Ractor.make_shareable(Proc.new{ a.length ... })

It can be used with define_method which can invoke from ractors:

define_method(name, Ractor.make_shareable(Proc.new{ ... }))`

But it is too long.

There are implementations for (a), (b) and (c), but the API is not fixed, so there is no PR now.

I'm thinking to introduce (c)'s feature in Ractor.make_sharaeble(pr).
To use with define_method, maybe it should be more friendly. Ideally, new syntax is great.

There is no conclusion, and your comments are welcome.

Thanks,
Koichi

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0