Project

General

Profile

Actions

Feature #10600

closed

[PATCH] Queue#close

Added by djellemah (John Anderson) over 7 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Target version:
-
[ruby-core:66843]

Description

In a multiple-producer / multiple-consumer situation using blocking enq and deq, closing a queue cleanly is difficult. It's possible using a queue poison token, but unpleasant because either producers have to know how to match up number of poison tokens with number of consumers, or consumers have to keep putting the poison back into the queue which complicates testing for empty and not blocking on deq.

This patch (from trunk at b2a128f) implements Queue#close which will close the queue to producers, leaving consumers to deq the remaining items. Once the queue is both closed and empty, consumers will not block. When an empty queue is closed, all consumers blocking on deq will be woken up and given nil.

With Queue#close, clean queue shutdown is simple:

queue = SizedQueue.new 1000

consumer_threads = lots_of.times.map do
  Thread.new do
    while item = queue.pop
      do_work item
    end
  end
end

source = somewhat_async_enumerator

producer_threads = a_few.times.map do
  Thread.new do
    loop{queue << source.next}
  end
end

producer_threads.each &:join
queue.close
consumer_threads.each &:join

Files

queue-close.diff (5.18 KB) queue-close.diff djellemah (John Anderson), 12/15/2014 09:10 AM
queue-close-2.diff (10.2 KB) queue-close-2.diff djellemah (John Anderson), 12/17/2014 05:38 PM
patch-25f99aef.diff (25.2 KB) patch-25f99aef.diff djellemah (John Anderson), 02/25/2015 07:51 PM
queue_benchmark.rb (2.95 KB) queue_benchmark.rb djellemah (John Anderson), 03/25/2015 07:30 PM

Related issues 1 (1 open0 closed)

Related to Ruby master - Feature #17357: `Queue#pop` should have a block form for closed queuesOpenActions
Actions

Also available in: Atom PDF