Project

General

Profile

Actions

Feature #11817

open

map.parallel

Added by eike.rb (Eike Dierks) almost 6 years ago. Updated almost 6 years ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:72120]

Description

I'd like to suggest a new method: #parallel

That method should take care of executing workloads in parallel across multiple cores.

I believe that #map can sometimes be distributed to many simultaneous threads.

So we would spread out the map blocks into parallel processing,
and then collect the results in order.

I believe this would need some new infrastructure like apple did with great_central,
so I'm talking along these lines.
(Aka we only want to open up as many threads as there are cores)

This would not help much with a simple map (because we need to collect and serialize the results in the end)
but it could be cool for more complex mappings.

Maybe this should have been a rails topic first,
because there it might make a lot of sense to render partials in parallel,
this could really speed up rails response rendering ;-)

But I believe this is a topic for ruby in general.
Given that we now go to really many cores,
it should be at the heart of ruby to use that.

For a start we could just declare #parallel as -> return self.
and than improve on it.

A naive implementation would just open a new thread for every parallel map block,
but that would not work well -- it actually would be very slow.

So we actually need a thread management,
along the lines of great central.

I believe this should be built into the very core libs of the ruby language.

I know this is hard stuff, but it needs to be done.
Let's go parallel from the heart of the language.


Please let us discuss if adding the #parallel method would be a good idea.
Let's come up with some implementations to find the problems.

I'm explicitly referring to map#parallel here.
In rails that might be cache.parallel (if you followed)

for some math tasks it might also be
parallel do {}

Actions

Also available in: Atom PDF