Ruby » Backport87

It's entirely possible this is a duplicate but after 10 minutes searching via both the Redmine interface and Google focused on this site I couldn't find it already reported.

Anything else I can do to help this along?

I thought this was fixed in 1.8.7
Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
That might help.

> I thought this was fixed in 1.8.7
> Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?

I just ran the same tests on 1.8.6 and 1.8.7.  It's still not fixed.  I'm not sure where the source code for 1.8.8 is so I didn't test it.

The build information and test results are below.  I didn't test 1.8.7 with --disable-ucontext because that configure flag doesn't seem to exist anymore.

Admittedly, ruby 1.8.7 is much faster than 1.8.6, but 1.8.7 still has this bug.

I would be happy to test 1.8.8 if you would tell me where to get the source code.

------------

1.8.6 with pthreads test results:
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
# autoconf && ./configure --enable-pthread && make
# ./ruby -v
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real	0m6.462s
user	0m6.128s
sys	0m0.308s

------------

1.8.6 with pthreads enabled and ucontext disabled
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
# autoconf && ./configure --disable-ucontext --enable-pthread && make
# ./ruby -v
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real	0m5.582s
user	0m5.548s
sys	0m0.000s

------------

1.8.7 with pthreads
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
# autoconf && ./configure --enable-pthread && make
# ./ruby -v
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real	0m3.241s
user	0m2.960s
sys	0m0.260s

------------

1.8.7 with --disable-pthread
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
# autoconf && ./configure --disable-pthread && make
# ./ruby -v
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real	0m2.535s
user	0m2.512s
sys	0m0.004s

ruby 1.8.8 have the same issue.

$ autoconf && ./configure --disable-pthread && make 

$  time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]

real    0m1.176s
user    0m1.171s
sys     0m0.005s


$ autoconf && ./configure --enable-pthread && make

[kosaki@kosaopt ruby_1_8]$  time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]

real    0m1.513s
user    0m1.340s
sys     0m0.173s

$ strace -c  ./ruby -e '1_000_000.times { x = 2 ** 256 }'
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 99.98    0.160160           0   1000011           rt_sigprocmask
  0.02    0.000025           2        14           mprotect
  0.00    0.000000           0         8           read
  0.00    0.000000           0        19        10 open
  0.00    0.000000           0         9           close
  0.00    0.000000           0         4         3 stat
  0.00    0.000000           0         9           fstat
  0.00    0.000000           0        26           mmap
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         7           brk
  0.00    0.000000           0        14           rt_sigaction
  0.00    0.000000           0         1         1 access
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         6           getrlimit
  0.00    0.000000           0         1           getuid
  0.00    0.000000           0         1           getgid
  0.00    0.000000           0         2           geteuid
  0.00    0.000000           0         2           getegid
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0         2         1 futex
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         1           set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00    0.160185               1000141        15 total


summary of ruby_1_8 code

  node.h
  ------------------------------------------
  #if defined(HAVE_GETCONTEXT) && defined(HAVE_SETCONTEXT)
  #include <ucontext.h>
  #define USE_CONTEXT
  #endif
  #include <setjmp.h>
  #include "st.h"

The syscam can use getcontext(), USE_CONTEXT is always set.

  eval.c
  ------------------------------------------
  #ifdef USE_CONTEXT
  (snip)
  #  define ruby_setjmp(just_before_setjmp, j) ((j)->status = 0, \
       (just_before_setjmp), \
       PRE_GETCONTEXT, \
       getcontext(&(j)->context), \
       POST_GETCONTEXT, \
       (j)->status)

if USE_CONTEXT is defined, ruby_setjmp() mean to call getcontext().
and linux getcontext() implementation call rt_sigprocmask() internally.

note: 1.9.x tree don't have this issue beucause ruby_setjmp() don't use getcontext().

This seems to be introduced following commit.

commit 9fdbc41973971f4ef032d57c4e02ed0430227a0a
Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
Date:   Sat Dec 13 00:01:28 2003 +0000

    * configure.in: check ucontext.h.

    * eval.c: use getcontext/setcontext() instead of setjmp/longjmp()
      on ia64 or with native thread enabled.  [ruby-core:01932]


    git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@5180 b2dd03c8-39d4-4d8f-98ff-823fe69b080e


but unfortunatelly, [ruby-core:01932] has alomost zero information ;)

  http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/1932

Thanks for looking into this one.
-r

I and nobu digged previous discussion awhile. Now, we have concluded getcontext() is only necessary when following cases
  1) Using linuxthreads (it has broken signal model)
  2) cpu arch has register stack

Any chance of backporting the fix to the ruby_1_8_7 branch?

Thanks,
Andre

There is.
But unfortunately 1.8.7-p299 was released very recently. So you need to wait about half year ;-)
Also, If anyone will find a bug in this patch, our chance will disappear....

Oh, sorry for the inconvenience... I was not aware of this.  It should be backported.

Hello

I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:

if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null`"; then
    use_context=yes
fi

Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does *not* appear in that output?

Apologies if I have misunderstood the patch.

Best regards,
Andre

2010/7/10 Andre Nathan <redmine@ruby-lang.org>:
> Issue #2553 has been updated by Andre Nathan.
>
>
> Hello
>
> I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
>
> if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null`"; then
>    use_context=yes
> fi
>
> Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does *not* appear in that output?
>
> Apologies if I have misunderstood the patch.

Oops, this patch only works on 32bit machine, because some 64bit
distro don't have /lib/libc.so.6 (instead they use /lib64), and this
condition select use_context=yes.

I'll update this soon.
And, if you are using 32bit distro, please show your /lib/libc.so.6
execution result.

I'm very sorry this.

2010/7/10 KOSAKI Motohiro <kosaki.motohiro@gmail.com>:
> 2010/7/10 Andre Nathan <redmine@ruby-lang.org>:
>> Issue #2553 has been updated by Andre Nathan.
>>
>>
>> Hello
>>
>> I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
>>
>> if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null`"; then
>>    use_context=yes
>> fi
>>
>> Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does *not* appear in that output?
>>
>> Apologies if I have misunderstood the patch.
>
> Oops, this patch only works on 32bit machine, because some 64bit
> distro don't have /lib/libc.so.6 (instead they use /lib64), and this
> condition select use_context=yes.
>
> I'll update this soon.
> And, if you are using 32bit distro, please show your /lib/libc.so.6
> execution result.
>
> I'm very sorry this.

following patch works both 32bit and 64bit on my environment.

$ svn diff
Index: configure.in
===================================================================
--- configure.in        (リビジョン 28594)
+++ configure.in        (作業コピー)
@@ -1151,7 +1151,7 @@
 if test x"$rb_with_pthread" = xyes; then
     AS_CASE("$target_cpu:$target_os:$cross_compiling",
     [*:linux*:no], [
-        if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native
POSIX Threads') 2> /dev/null`"; then
+        if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep
'linuxthreads') 2> /dev/null`"; then
            use_context=yes
        fi
     ],

> following patch works both 32bit and 64bit on my environment.

Committed.
Revision 28595 and 28597.

Thanks Andre, you are great.

The new patch fixed this issue for me. Thanks a lot!

Here's hoping that the patch will be merged into ruby_1_8_7. I believe it'll make it easier for distributions who package this version to have this performance bug fixed.

Thanks again!
Andre