Feature #17100
closedRactor: a proposal for a new concurrent abstraction without thread-safety issues
Description
Ractor: a proposal for a new concurrent abstraction without thread-safety issues¶
Abstract¶
This ticket proposes a new concurrent abstraction named "Ractor," Ruby's Actor-like feature (not an exact Actor-model).
Ractor achieves the following goals:
- Parallel execution in a Ruby interpreter process
- Avoidance of thread-safety issues (especially race issues) by limiting object sharing
- Communication via copying and moving
I have been working on this proposal for a few years. The project name has been "Guild," but I renamed it to Ractor following Matz' preference.
Resources:
- Proposed specification: https://github.com/ko1/ruby/blob/ractor_parallel/doc/ractor.md
- My talk:
- (latest, but written in Japanese) http://atdot.net/~ko1/activities/2020_ruby3summit.pdf
- (old, API is changed) http://atdot.net/~ko1/activities/2018_rubykaigi2018.pdf
- (old, API is changed) http://atdot.net/~ko1/activities/2018_rubyconf2018.pdf
Current implementation is not complete (contains many bugs) but it passes the current CI. I propose to merge it soon and try the API, and to continue working on the implementation on master branch.
Background¶
MRI doesn't provide an in-process parallel computation feature because parallel "Threads" have many issues:
- Ruby programmers need to consider about Thread-safety more.
- Interpreter developers need to consider about Thread-safety more.
- Interpreter will slow down in single thread execution because of fine-grain synchronization without clever optimizations.
The reason for these issues is "shared-everything" thread model.
Proposal¶
To overcome the issues on multiple-threads, Ractor abstraction is proposed. This proposal consists of two layers: memory model and communication model.
Basics:
- Introduce "Ractor" as a new concurrent entity.
- Ractors run in parallel.
Memory model:
- Separate "shareable" objects and "un-shareable" objects among ractors running in parallel.
- Shareable objects:
- Immutable objects (frozen objects only refer to shareable objects)
- Class/module objects
- Special shareable objects (Ractor objects, and so on)
- Un-shareable objects:
- Other objects
- Shareable objects:
- Most objects are "un-shareable," which means we Ruby programmers and interpreter developers don't need to care about thread-safety in most cases.
- We only concentrate on synchronizing "shareable" objects.
- Compared with completely separated memory model (like MVM proposal), programming will be easier.
- This model is similar to Racket's
Place
abstraction.
Communication model:
- Actor-like (not the same) message passing using
Ractor#send(obj)
andRactor.recv
- Pull-type communication using
Ractor.yield(obj)
andRactor#take
- Support for multiple waiting using
Ractor.select(...)
Actor-like model is the origin of the name of our proposal "Ractor" (Ruby's actor). However, currently, it is not an Actor model because we can't select the message (using pattern match as in Erlang, Elixir, ...). This means that we can't have multiple communication channels. Instead of adopting an incomplete actor model, this proposal provides yield
/take
pair to handle multiple channels. We discuss this topic later.
I strongly believe the memory model is promising. However, I'm not sure if the communication model is the best. This is why I introduced "experimental" warning.
Proposed specification: https://github.com/ko1/ruby/blob/ractor_parallel/doc/ractor.md
Implementation¶
https://github.com/ruby/ruby/pull/3365
All GH actions pass.
I describe the implementation briefly.
rb_ractor_t
¶
Without Ractor, the VM-Thread-Fiber hierarchy is like this:
- The VM
rb_vm_t
manages running threads (rb_thread_t
). - A thread (
rb_thread_t
) points to a running fiber (rb_fiber_t
).
With Ractor, we introduce a new layer rb_ractor_t
:
- The VM
rb_vm_t
manages running ractors (rb_ractor_t
). - A Ractor manages running threads (
rb_thread_t
). - A thread (
rb_thread_t
) points to a running fiber (rb_fiber_t
).
rb_ractor_t
has a GVL to manage threads (only one among a Ractor's threads can run).
Ractor implementation is located in ractor.h
, ractor.c
and ractor.rb
.
VM-wide lock¶
VM-wide lock is introduced to protect VM global resources such as object space. It should allow recursive lock, so the implementation is a monitor. We shall call it VM-wide monitor. For now, RB_VM_LOCK_ENTER()
and RB_VM_LOCK_LEAVE()
are provided to acquire/release the lock.
Note that it is different from the (current) GVL. A GVL is acquired anytime you run a Ruby thread. VM-wide lock is acquired only when accessing VM-wide resources.
On single ractor mode (all Ruby scripts except my tests)
Object management and GC¶
- (1) All ractors share the object space.
- (2) Each GC event will stop all ractors, and a ractor GC works under barrier synchronization.
- Barrier at
gc_enter()
- marking, (lazy) sweeping, ...
- Barrier at
- (3) Because all of the object space are shared by ractors, object creation is protected by VM-wide lock.
(2) and (3) have huge impact on performance. The plan is:
- For (2), introduce (semi-)separated object space. It would require a long time and Ruby 3.0 can't employ this technique.
- For (3), introduce free slot cache for every ractor; then most creations can be done without synchronization. It will be employed soon.
Experimental warning¶
Currently, Ractor implementation and specification are not stable. So upon its first usage, Ractor.new
will show a warning:
warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.
Discussion¶
Actor-based and channel-based¶
I think there are two message passing approaches: Actor-based (as in Erlang, ...) and channel-based (as in Go, ...).
With channel-based approach, it is easy to manipulate multiple channels because it manages them explicitly. Actor-based approach manipulates multiple channels with message pattern. The receiver can ignore unexpected structured messages or put them on hold and can handle them after the behavior has changed (role of actor has changed).
Ractor has send/recv
like Actor-model, but there is no pattern matching feature. This is because we can't introduce new syntax, and I can't design a good API.
With channel-based approach, it is easy to design the API (for example, do ch = Ractor::Channel.new
and share the ch
that ractors can provide). However, I can't design a good API to handle exceptions among Ractors.
Regarding error handling, we propose a hybrid model using send/recv
, yield/take
pairs. Ractor#take
can receive the source ractor's exception (like Thread#join
). On Actor approach, we can detect when the destination Ractor is not working (killed) upon Ractor#send(obj)
. A receiver ractor (waiting for Ractor.recv
) cannot detect the sender's trouble, but maybe the priority is not high. Ractor#take
also detects the sender's (Ractor.yield(obj)
) error, so the error can be propagated.
To handle multiple communication channels on Ractor, instead of using multiple channels, we use pipe ractors.
# worker-pool (receive by send)
main # pipe.send(obj)
-> pipe # Ractor.yield Ractor.recv
->
worker1 # Ractor.yield(some_task pipe.take))
worker2 # Ractor.yield(some_task pipe.take))
worker3 # Ractor.yield(some_task pipe.take))
-> main # Ractor.select(worker1, worker2, worker3)
# if worker* causes an error, main can detect the error.
pipe ractors may look like channels. However, we don't need to introduce new classes with this technique (the implementation can omit Ractor creation for pipe ractors).
Maybe there are other possibilities. For example, if we can propagate the errors with channels, we can also consider a channel-model (we need to change the Ractor name :p then).
Name of Ractor (and Guild)¶
When I proposed Guild in 2016, I regarded "move" message-passing (see specification) to be characteristic of it, and I called this feature "moving membership." This is why the name "Guild" was chosen. However Matz pointed out that this move semantics is not used frequently, and he asked me to change the name. Also, someone had already been using the class name "Guild."
"Ractor" is short and is not an existing class; this is why I choose "Ractor."
I understand people may confuse it with "Reactor."
TODO¶
There are many remaining tasks.
Protection¶
Many VM-wide (process-wide) resources are not protected correctly, so using Ractor on a complicated program can cause critical bugs ([BUG]
). Most global resource are managed by global variables, so we need to check them correctly.
C-methods¶
Currently, C-methods (methods written in C and defined with rb_define_method()
) run in parallel. It means that thread-unsafe code can run in parallel. To solve this issue, I plan the following:
(1) Introduce thread-unsafe label for methods
It is impossible to make all C-methods thread-safe, especially for C-methods in third party C-extensions. To protect them, label these (possibly) thread-unsafe C-methods as "thread-unsafe."
When "unsafe"-labeled C methods are invoked, they acquire a VM-wide lock. This VM-wide lock should check for recursiveness (so this lock should be a monitor) and escaping (exceptions). Currently, VM-wide lock doesn't check for escaping, but that should be implemented soon.
(2) Built-in C-methods
I'll fix most of the builtin C-methods (String, Array, ...) so that they will become thread-safe. If it is not easy, I'll use thread-unsafe label.
Copying and moving¶
Currently, Marshal protocol makes deep copy on message communication. However, Marshal protocol doesn't support some objects like Ractor
objects, so we need to modify them.
Only a few types are supported for moving, so we need to write more.
"GVL" naming¶
Currently, the source code contains the name "GVL" for Ractor local locks. Maybe they should be renamed.
Performance¶
To introduce fine-grained lock, performance tuning is needed.
Bug fixes¶
many many ....
Conclusion¶
This ticket proposes a new concurrent abstraction "Ractor." I think Ruby 3 can ship with Ractor under "experimental" status.