Bug #1169: Thread#priority still doesn't work

Native threaded implementations will have a difficult time implementing the
same semantics as the green threaded Thread.priority=
One would probably have to resort to a "real-time" thread classes.
Under Linux, I'm pretty sure that would require the Ruby process have root
privileges.

There has been some discussion of deprecating Thread.priority= for this
reason.

- brent



Michal Babej-2 wrote:
> 
> Bug #1169: Thread#priority still doesn't work
> http://redmine.ruby-lang.org/issues/show/1169
> 
> Author: Lucas Nussbaum
> Status: Open, Priority: Normal
> ruby -v: 1.9.1p0
> 
> Hi,
> 
> In [ruby-dev:33124], it was reported that Thread#priority didn't work
> properly. Apparently, it still doesn't work: test_priority still randomly
> fails during 'make tests-all'.
> 
> Example script:
> c1 = c2 = 0
> t1 = Thread.new { loop { c1 += 1 } }
> t1.priority = -1
> t2 = Thread.new { loop { c2 += 1 } }
> t2.priority = -3
> sleep 5
> t1.kill
> t2.kill
> puts "#{c1} #{c2} #{c1 > c2}"
> 
> Output:
> $ while true; do ruby1.9 t.rb ; done
> 38282412 29165036 true
> 62366432 412710 true
> 11895940 263708221 false
> 24823756 62805923 false
> 5256995 68944142 false
> 49038072 14617852 true
> 47802027 28211284 true
> 17908514 62888354 false
> 36634663 68310354 false
> 24960080 39373933 false
> 
> 
> ----------------------------------------
> http://redmine.ruby-lang.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/-ruby-core%3A22208---Bug--1169--Thread-priority-still-doesn%27t-work-tp22075054p22088731.html
Sent from the ruby-core mailing list archive at Nabble.com.

• Assignee set to ko1 (Koichi Sasada)
• Priority changed from Normal to Low
• Target version set to 2.0.0

Data point: This fails about 99% of the time in Solaris on sparc64.

• Status changed from Open to Closed
• Target version changed from 2.0.0 to 1.9.2

Hi,

The problematic test was disabled at r26155.

And now, rdoc of Thread#priority was refined at r27351.
It is just a hint for thread scheduler, which may be ignored on
some platforms.

-- 
Yusuke Endoh <[e-mail]>

So, you've disabled the test which demonstrates the problem. You've documented that the desired behavior is now going to be optional. And now, you've closed the ticket which reported the problem. However:

          >>>the bug still exists<<<

I do not think this issue should have been closed. Can it be reopened, please? Open bug reports are the collective memory about where the problems in a system are. When you close a report without fixing the reported bug, it's like saying, "We're just going to forget that this problem was ever noticed." But it doesn't make the problem go away; it just makes it harder to know that it's there.

Should thread priorities be respected or not? If they ought to be respected, then it is better to have a note somewhere that keeps track of the fact that they may not be working. I say it's better to leave unresolved bug reports open than to let the bug tracker reflect a false state of affairs.  

I don't think you need access to the realtime scheduling algorithms (SCHED_RR, SCHED_FIFO) to get higher priority threads to receive more cputime. Normal (fair) scheduling (SCHED_OTHER) should be sufficient. After all, the syscalls nice(2), setpriority(2), and pthread_setschedparam(3) presumably work for non-realtime processes/threads.

However, I'm now starting to theorize that there's an interaction here between the scheduler and the GIL which causes priorities to be effectively ignored. Imagine this:

2 threads running, A and B; A.priority > B.priority
initially, A runs (so it holds GIL)
B is waiting on GIL 
after a while (10ms I think) scheduler forces a context switch
so now A unlocks GIL, then immediately relocks it (yielding time to other threads)
but before A can relock, B is scheduled, since it was waiting. Now B owns GIL.
after another 10ms, B yields time back to A
...and so on

So, A and B are effectively alternating timeslices, each getting roughly equal amounts of time, even tho A should have a higher priority. If the os uses priority queues rather than normal fifo queues for the internal queue inside a mutex, then is problem wouldn't occur. (Actually, you need to have more than 2 threads in your system for a priority queue to be helpful here, because of a race condition, but never mind that detail.) However, I think it's quite unlikely that typical desktop or server OSes use priority queues here; an RTOS might use them (I know vxworks has it as an option, or used to anyway) but priority queues require more cpu and memory and are at least tricky (maybe impossible) to implement right in a system with a fair scheduler. (Fair schedulers are actually pretty complicated compared to realtime schedulers.)

So far, this is all just a theory. I have no proof either for the overall theory or my contention that the queue used inside a mutex is (usually, on most systems) a fifo queue. 

If I'm right, then this might be fixable by making timeslices variable length, instead of always 10 ms. Lower priority threads would get shorter timeslices. 

Hi,

>          >>>the bug still exists<<<

No.  The spec was changed so that Thread#priority was just a
hint for thread scheduler.  So, the bug is no longer a bug.

I guess that it is too restrictive for implementation to make
exact support for Thread#priority mandatory.

And generally, it is bad practice to rely on thread priority
in your design.  We should think thread priority as a guide
to efficiency, at most.


> this might be fixable by making timeslices variable length, instead of always 10 ms. Lower priority threads would get shorter timeslices.

Interesting.

I doubt whether the feature is portable because it heavily
depends on granularity of wait function of each platform, but
it might be work better on many popular platform.

But it is feature request.
Could you register to Feature tracker new ticket with clearer
explanation?

-- 
Yusuke Endoh <[e-mail]>