Bug #9432: ThreadError [ Attempt to unlock a mutex which is locked by another thread ]

[rajesh shanmugam]

I use ruby-2.0.0-p247. I seem to get this issue frequently in threaded environment. (Sidekiq)

I am not very sure if it a ruby thread issue as such or something I am doing wrong. If there is any more details you need I would be happy to provide you.

Operating system: Ubuntu

Trace

/home/ubuntu/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/thread.rb:188:in `synchronize'
/home/ubuntu/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/thread.rb:188:in `block in pop'
/home/ubuntu/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/thread.rb:187:in `handle_interrupt'
/home/ubuntu/.rvm/rubies/ruby-2.0.0-p247/lib/ruby/2.0.0/thread.rb:187:in `pop'

Regards
Rajesh

[Eric Wong]

I use ruby-2.0.0-p247. I seem to get this issue frequently in threaded
environment. (Sidekiq)

I am not very sure if it a ruby thread issue as such or something I am
doing wrong. If there is any more details you need I would be happy to
provide you.

Have you contacted the sidekiq author about this?

[Tomoyuki Chikanaga]

Hello, rajesh.

Thank you for your report.
Could you try to reproduce it with 2.1 and/or trunk?

[Aaron Stone]

Note: Sidekiq does not appear to be using eventmachine, which has a similar bug at #9132. Any ideas on what is going on here? I can reproduce the bug from all releases of Ruby 2.0.0pxxx including 2.0.0p598.

[Eric Wong]

Can you show us a small test case? Which version of Sidekiq?

Note: this issue seems to not affect Ruby 2.1+ because thread.rb was
rewritten in C. I looked briefly and #9132 but I'd rather deal
with issues in pure Ruby or C, not C++.

[Hiroshi SHIBATA]

• ruby -v set to 2.0.0
• Project changed from Backport200 to Ruby trunk
• Tracker changed from Backport to Bug

[Aaron Stone]

The error also shows up here: https://github.com/iconara/cql-rb/issues/68

This is not an issue with the applications or the gems, or that eventmachine is written in C++. It's an MRI Ruby problem in the 2.0 implementation of Queue.pop, which is different than the 1.9.3 implementation that does not have this problem.

Ruby 2.0 is a widely deployed and supported version.
I'd really appreciate if someone upstream would take this bug report seriously.

[Aaron Stone]

In Ruby 1.9.3, thread.rb has Queue.pop defined as:

183   def pop(non_block=false)
184     @mutex.synchronize{
185       while true
186         if @que.empty?
187           raise ThreadError, "queue empty" if non_block
188           @waiting.push Thread.current
189           @mutex.sleep
190         else
191           return @que.shift
192         end
193       end
194     }
195   end

In Ruby 2.0, thread.rb has Queue.pop defined as:

186   def pop(non_block=false)
187     Thread.handle_interrupt(StandardError => :on_blocking) do
188       @mutex.synchronize do
189         while true
190           if @que.empty?
191             if non_block
192               raise ThreadError, "queue empty"
193             else
194               begin
195                 @num_waiting += 1
196                 @cond.wait @mutex
197               ensure
198                 @num_waiting -= 1
199               end
200             end
201           else
202             return @que.shift
203           end
204         end
205       end
206     end
207   end

The use of ConditionVariable is new in 2.0 vs. 1.9.3, and I think it is causing the problem. In 1.9.3, Queue.push itself walks its @waiting array to find and wake up a thread that will consume the just-pushed element. In 2.0, Queue.push calls @cond.signal which then goes and looks for a thread to wake up.

Ok, here's the most salient difference:
In 1.9.3, Queue.push calls thread.wakeup on a consuming thread.
In 2.0, Queue.push calls ConditionVariable.signal calls thread.run on a consuming thread.

The difference is that thread.run also executes the scheduler immediately.

In Ruby 2.1 and up, thread.rb does not exist - it appears to be rewritten in C.

[Aaron Stone]

Dear Ruby maintainers: this is still a problem. Please, help out here.