Feature #18965
closed
Further Thread::Queue improvements
Description
Following the recent addition of a timeout parameter to Queue#pop, there are a handful of other improvements I'd like to make.
Batch insert¶
When using the queue for batch processing, it would be good to be able to push multiple elements at once:
Currently you have to call push repeatedly
items.each do |item|
queue.push(item)
end
That's wasteful because on each call we check wether the queue is closed, try to wakeup blocked threads, etc.
It would be much better if you could do:
queue.concat(items)
With of course both nonblock and timeout support.
Then there's the question of how SizedQueue would behave if it's not full, but still doesn't have space for all the elements. e.g.
queue = SizedQueue.new(10)
queue.concat(6.times.to_a)
queue.concat(6.times.to_a) # Block until there is 6 free slots?
I think the simplest would be to wait for enough space to append the entire set, because combined with a timeout, it would be awkward if only part of the array was concatenated.
Batch pop¶
Similarly, sometimes the consumer of a queue is capable of batching, and right now it's not efficient:
loop do
items = [queue.pop]
begin
99.times do
items << queue.pop(true) # true is for nonblock
end
rescue ThreadError # empty queue
end
process_items(items)
end
It would be much more efficient if pop accepted a count parameter:
loop do
items = queue.pop(count: 100)
process_items(items)
end
The behavior would be:
- Block if the queue is empty
- If it's not empty, return up to
countitems (Just likeArray#pop)
Non blocking mode, without exception¶
As shown above, the current nonblock parameter is a bit awkward, because:
- It raises an exception, which is very expensive for a construct often used in "low level" code.
- The exception is
ThreadError, so you may have to match the error message for"queue empty", to make sure it doesn't come from a Mutex issue or something like that.
I believe that we could introduce a keyword argument:
Queue.new.pop(nonblock: true) # => nil
Updated by 
Updated by 
Updated by 
Updated by