Bug #1169: Thread#priority still doesn't work

• Status: Closed
• Assignee: ko1 (Koichi Sasada)
• Target version: 1.9.2
• ruby -v: 1.9.1p0

[lucas (Lucas Nussbaum)]

=begin
Hi,

In [ruby-dev:33124], it was reported that Thread#priority didn't work properly. Apparently, it still doesn't work: test_priority still randomly fails during 'make tests-all'.

Example script:
c1 = c2 = 0
t1 = Thread.new { loop { c1 += 1 } }
t1.priority = -1
t2 = Thread.new { loop { c2 += 1 } }
t2.priority = -3
sleep 5
t1.kill
t2.kill
puts "#{c1} #{c2} #{c1 > c2}"

Output:
$ while true; do ruby1.9 t.rb ; done
38282412 29165036 true
62366432 412710 true
11895940 263708221 false
24823756 62805923 false
5256995 68944142 false
49038072 14617852 true
47802027 28211284 true
17908514 62888354 false
36634663 68310354 false
24960080 39373933 false
=end

[brent (Brent Roman)]

=begin

Native threaded implementations will have a difficult time implementing the
same semantics as the green threaded Thread.priority=
One would probably have to resort to a "real-time" thread classes.
Under Linux, I'm pretty sure that would require the Ruby process have root
privileges.

There has been some discussion of deprecating Thread.priority= for this
reason.

• brent

Michal Babej-2 wrote:

Bug #1169: Thread#priority still doesn't work
http://redmine.ruby-lang.org/issues/show/1169

Author: Lucas Nussbaum
Status: Open, Priority: Normal
ruby -v: 1.9.1p0

Hi,

In [ruby-dev:33124], it was reported that Thread#priority didn't work
properly. Apparently, it still doesn't work: test_priority still randomly
fails during 'make tests-all'.

Example script:
c1 = c2 = 0
t1 = Thread.new { loop { c1 += 1 } }
t1.priority = -1
t2 = Thread.new { loop { c2 += 1 } }
t2.priority = -3
sleep 5
t1.kill
t2.kill
puts "#{c1} #{c2} #{c1 > c2}"

Output:
$ while true; do ruby1.9 t.rb ; done
38282412 29165036 true
62366432 412710 true
11895940 263708221 false
24823756 62805923 false
5256995 68944142 false
49038072 14617852 true
47802027 28211284 true
17908514 62888354 false
36634663 68310354 false
24960080 39373933 false

---

http://redmine.ruby-lang.org

--
View this message in context: http://www.nabble.com/-ruby-core%3A22208---Bug--1169--Thread-priority-still-doesn%27t-work-tp22075054p22088731.html
Sent from the ruby-core mailing list archive at Nabble.com.

=end

[yugui (Yuki Sonoda)]

• Assignee set to ko1 (Koichi Sasada)
• Priority changed from Normal to 3
• Target version set to 2.0.0

=begin

=end

[docwhat (Christian Höltje)]

=begin
Data point: This fails about 99% of the time in Solaris on sparc64.
=end

[mame (Yusuke Endoh)]

• Status changed from Open to Closed
• Target version changed from 2.0.0 to 1.9.2

=begin
Hi,

The problematic test was disabled at r26155.

And now, rdoc of Thread#priority was refined at r27351.
It is just a hint for thread scheduler, which may be ignored on
some platforms.

--
Yusuke Endoh [e-mail]
=end

[coatl (caleb clausen)]

=begin
So, you've disabled the test which demonstrates the problem. You've documented that the desired behavior is now going to be optional. And now, you've closed the ticket which reported the problem. However:

       >>>the bug still exists<<<

I do not think this issue should have been closed. Can it be reopened, please? Open bug reports are the collective memory about where the problems in a system are. When you close a report without fixing the reported bug, it's like saying, "We're just going to forget that this problem was ever noticed." But it doesn't make the problem go away; it just makes it harder to know that it's there.

Should thread priorities be respected or not? If they ought to be respected, then it is better to have a note somewhere that keeps track of the fact that they may not be working. I say it's better to leave unresolved bug reports open than to let the bug tracker reflect a false state of affairs.

I don't think you need access to the realtime scheduling algorithms (SCHED_RR, SCHED_FIFO) to get higher priority threads to receive more cputime. Normal (fair) scheduling (SCHED_OTHER) should be sufficient. After all, the syscalls nice(2), setpriority(2), and pthread_setschedparam(3) presumably work for non-realtime processes/threads.

However, I'm now starting to theorize that there's an interaction here between the scheduler and the GIL which causes priorities to be effectively ignored. Imagine this:

2 threads running, A and B; A.priority > B.priority
initially, A runs (so it holds GIL)
B is waiting on GIL
after a while (10ms I think) scheduler forces a context switch
so now A unlocks GIL, then immediately relocks it (yielding time to other threads)
but before A can relock, B is scheduled, since it was waiting. Now B owns GIL.
after another 10ms, B yields time back to A
...and so on

So, A and B are effectively alternating timeslices, each getting roughly equal amounts of time, even tho A should have a higher priority. If the os uses priority queues rather than normal fifo queues for the internal queue inside a mutex, then is problem wouldn't occur. (Actually, you need to have more than 2 threads in your system for a priority queue to be helpful here, because of a race condition, but never mind that detail.) However, I think it's quite unlikely that typical desktop or server OSes use priority queues here; an RTOS might use them (I know vxworks has it as an option, or used to anyway) but priority queues require more cpu and memory and are at least tricky (maybe impossible) to implement right in a system with a fair scheduler. (Fair schedulers are actually pretty complicated compared to realtime schedulers.)

So far, this is all just a theory. I have no proof either for the overall theory or my contention that the queue used inside a mutex is (usually, on most systems) a fifo queue.

If I'm right, then this might be fixable by making timeslices variable length, instead of always 10 ms. Lower priority threads would get shorter timeslices.
=end

[mame (Yusuke Endoh)]

=begin
Hi,

     >>>the bug still exists<<<

No. The spec was changed so that Thread#priority was just a
hint for thread scheduler. So, the bug is no longer a bug.

I guess that it is too restrictive for implementation to make
exact support for Thread#priority mandatory.

And generally, it is bad practice to rely on thread priority
in your design. We should think thread priority as a guide
to efficiency, at most.

this might be fixable by making timeslices variable length, instead of always 10 ms. Lower priority threads would get shorter timeslices.

Interesting.

I doubt whether the feature is portable because it heavily
depends on granularity of wait function of each platform, but
it might be work better on many popular platform.

But it is feature request.
Could you register to Feature tracker new ticket with clearer
explanation?

--
Yusuke Endoh [e-mail]

=end