Feature #8793: Ruby 2.0 and Threads under HPUX

• Status: Closed
• Assignee: -
• Target version: -

[[e-mail] (Michal Rokos)]

Use of Threads under HPUX currently raise rb_eNotImpError ("ruby engine can initialize only in the main thread") since the STACKADDR_AVAILABLE is not available.

This brings basic support for the get_stack() under HPUX.

This patch also resolves issue under HPUX where signals usually cause the coredump since the stack size is too small.

It seems that the patch is valid not only for Ruby 2.0, but also for Ruby 1.9.

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Feedback

Does pthread_attr_init() set the current thread info?

[[e-mail] (Michal Rokos)]

Nope, pthread_attr_init() populates default values.

[nobu (Nobuyoshi Nakada)]

• Status changed from Feedback to Closed
• % Done changed from 0 to 100

This issue was solved with changeset r42716.
Michal, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

thread_pthread.c: get_stack on HP-UX

• thread_pthread.c (hpux_attr_getstackaddr): basic support for the
  get_stack() under HP-UX. based on the patch by [e-mail]
  (Michal Rokos) at [ruby-core:56645]. [Feature #8793]

[[e-mail] (Michal Rokos)]

Nobu, thank you for your effort. I am going to test trunk ruby soon and I will let you know.

[nobu (Nobuyoshi Nakada)]

Thank you.
I'm unsure about vps_pagesize and its default value, could you check it?

And, it might be better to use _pthread_stack_info_np(), but I couldn't find out its correct usage.
Don't you have any info, the signature, related types, return type, and others?
I found just that WebKit uses it, but I want not to read the source because I'm uncertain about its license.

[[e-mail] (Michal Rokos)]

On systems, I have a access to, the vps_pagesize is set to 16.

/usr/sbin/kctune -v vps_pagesize
Tunable vps_pagesize
Description Default user page size (kBytes)
Module vm
Current Value 16 [Default]
Value at Next Boot 16 [Default]
Value at Last Boot 16
Default Value 16
Constraints vps_pagesize >= 4
vps_pagesize <= 4194304
Can Change Immediately or at Next Boot

Regarding the _pthread_stack_info_np(), the API is
int _pthread_stack_info_np(pthread_t id, struct _pthread_stack_info *state);
and the _pthread_stack_info is
typedef struct _pthread_stack_info {
int stk_flags;

    /* Size of the stack */
    short   stk_stacksize_valid;
    size_t  stk_stacksize;

    /* Size of the guard page at the end of the stack */
    short   stk_guardsize_valid;
    size_t  stk_guardsize;

    /* Size of the Register Stack */
    short   stk_rsestacksize_valid;                                                                                                                       
    size_t  stk_rsestacksize;                                                                                                                             
                                                                                                                                                          
    /* Reserved for future use -- do not use, names WILL change */                                                                                        
    short   stk_reserved1_valid;                                                                                                                          
    size_t  stk_reserved1_size;                                                                                                                           
                                                                                                                                                          
    /* Address of the base of the stack */                                                                                                                
    void    *stk_stack_base;                                                                                                                              
                                                                                                                                                          
    /* Base of RSE BS.  Filled in only for IA64.  Do not use on PA. */                                                                                    
    void    *stk_rse_base;                                                                                                                                
                                                                                                                                                          
    /* Current stack pointer value */                                                                                                                     
    void    *stk_sp;                                                                                                                                      
                                                                                                                                                          
    /* Thread bsp.  Filled in only for IA64.  Do not use on PA. */                                                                                        
    void    *stk_bsp;                                                                                                                                     
                                                                                                                                                          
    /* Current program counter value */                                                                                                                   
    void    *stk_pc;                                                                                                                                      
                                                                                                                                                          
    /* Reserved for future use -- do not use, names WILL change */                                                                                        
    void    *stk_reserved[25];                                                                                                                            

} _pthread_stack_info_t;

But the obstacle with _pthread_stack_info_np() is that the polled thread 'id' has to be either stopped or suspended otherwise the _pthread_stack_info_np() is returning an error and does not provide values in 'state'. That is why i chose not to use is and use the _Asm_get_sp() instead.

[nobu (Nobuyoshi Nakada)]

• File 0001-thread_pthread.c-stack-info-on-HP-UX.patch 0001-thread_pthread.c-stack-info-on-HP-UX.patch added
• Status changed from Closed to Feedback

Thank you, and does this patch work?

[[e-mail] (Michal Rokos)]

Nobu, the merged patch (that is currently in the trunk) does not compile:

compiling thread.c
"thread_pthread.c", line 528: warning #2223-D: function "gettune" declared
          implicitly
        if (gettune("vps_pagesize", &pagesize)) {
            ^
"thread_pthread.c", line 534: warning #2171-D: invalid type conversion
      *addr = (void *)((size_t)((char *)_Asm_get_sp() - size) & ~(pagesize - 1));
              ^

"thread_pthread.c", line 534: error #2137: expression must be a modifiable
          lvalue
      *addr = (void *)((size_t)((char *)_Asm_get_sp() - size) & ~(pagesize - 1));
      ^
1 error detected in the compilation of "thread.c".

The error is due the missing * in the definition of hpux_attr_getstackaddr():

-hpux_attr_getstackaddr(const pthread_attr_t *attr, void *addr)
+hpux_attr_getstackaddr(const pthread_attr_t *attr, void **addr)

The gettune() is defined in <sys/dyntune.h> as int gettune(const char *tunable, uint64_t *value);

Otherwise the merged patch is OK.

[[e-mail] (Michal Rokos)]

nobu (Nobuyoshi Nakada) wrote:

Thank you, and does this patch work?

Patch compiles, but the 'make test' started to fail again, since the PTHREAD_STACK_MIN got back:

sample/test.rb:signal  
sendsig: useracc failed. 0x2000000077773e00 0x00000000005000
Pid 15617 was killed due to failure in writing the signal context - possible stack overflow.
#962 test_thread.rb:60:in `<top (required)>': core dumped  
#979 test_thread.rb:227:in `<top (required)>': core dumped  
#983 test_thread.rb:264:in `<top (required)>': 
     (0..10).map {
       Thread.new {
        10000.times {
           Object.new.to_s
         }
       }
     }.each {|t|
       t.join
     }
  #=> killed by SIGABRT (signal 6)
| (MR) thid 1: p = 7ffff000, s = 142602240, addr = 887fe000, err = 0
| (MR) thid 1: p = 7ffff000, s = 142602240, addr = 887fe000, err = 0
| (MR) thid 1: p = 7ffff000, s = 142602240, addr = 887fe000, err = 0
| (MR) thid 1: p = 7ffff000, s = 142602240, addr = 887fe000, err = 0
| (MR) thid 7: p = 77771000, s = 524288, addr = 777f1000, err = 0
| bootstraptest.tmp.rb:5: [BUG] object allocation during garbage collection phase
| ruby 2.1.0dev (2013-09-03 trunk 42801) [ia64-hpux11.31]
| 
| -- Control frame information -----------------------------------------------
| c:0005 p:---- s:0012 e:000011 CFUNC  :to_s
| c:0004 p:0015 s:0009 e:000008 BLOCK  bootstraptest.tmp.rb:5 [FINISH]
| c:0003 p:---- s:0007 e:000006 CFUNC  :times
| c:0002 p:0008 s:0004 e:000003 BLOCK  bootstraptest.tmp.rb:4 [FINISH]
| c:0001 p:---- s:0002 e:000001 TOP    [FINISH]
| 
| bootstraptest.tmp.rb:4:in `block (2 levels) in <main>'
| bootstraptest.tmp.rb:4:in `times'
| bootstraptest.tmp.rb:5:in `block (3 levels) in <main>'
| bootstraptest.tmp.rb:5:in `to_s'
| 
| -- Other runtime information -----------------------------------------------
| 
| * Loaded script: bootstraptest.tmp.rb
| 
| * Loaded features:
| 
|     0 enumerator.so
| 
| [NOTE]
| You may have encountered a bug in the Ruby interpreter or extension libraries.
| Bug reports are welcome.
| For details: http://bugs.ruby-lang.org/
| 
#984 test_thread.rb:276:in `<top (required)>': core dumped  [ruby-dev:34492]
#1002 test_thread.rb:399:in `<top (required)>': core dumped  
FAIL 5/50

Just to compare values between merged version and the Comment #7 version:

> ./miniruby -e 'Thread.new { sleep 5 }'
(MR) pthread_self 1: _Asm_get_sp() = 7fffe3c0, *addr = 7ffbc000, size = 262144 [thread_pthread.c:535]
(MR) addr = 7fffe400, stackaddr = 7fffc000, space = -9216, size = 262144 [thread_pthread.c:694]
(MR) pthread_self 1: _Asm_get_sp() = 7fffde90, *addr = 7ffbc000, size = 262144 [thread_pthread.c:535]
(MR) addr = 7fffded0, stackaddr = 7fffc000, space = -7888, size = 262144 [thread_pthread.c:694]
(MR) pthread_self 1: _Asm_get_sp() = 7fffde90, *addr = 7ffbc000, size = 262144 [thread_pthread.c:535]
(MR) addr = 7fffded0, stackaddr = 7fffc000, space = -7888, size = 262144 [thread_pthread.c:694]
(MR) pthread_self 1: _Asm_get_sp() = 7fffe3a0, *addr = 7ffbc000, size = 262144 [thread_pthread.c:535]
(MR) addr = 7fffe3e4, stackaddr = 7fffc000, space = -9188, size = 262144 [thread_pthread.c:694]
(MR) pthread_self 3: _Asm_get_sp() = 77778fc0, *addr = 77738000, size = 262144 [thread_pthread.c:535]
(MR) pthread_self 1: _Asm_get_sp() = 7fffd430, *addr = 7ffbc000, size = 262144 [thread_pthread.c:535]
(MR) addr = 7fffd474, stackaddr = 7fffc000, space = -5236, size = 262144 [thread_pthread.c:694]

and

> ./miniruby -e 'Thread.new { sleep 5 }'
(MR) thid 1: p = 7ffff000, s = 142602240, *addr = 887fe000, *size = 142602240 [thread_pthread.c:509]
(MR) addr = 7fffe400, stackaddr = 887fe000, space = 142605312, size = 142602240 [thread_pthread.c:707]
(MR) thid 1: p = 7ffff000, s = 142602240, *addr = 887fe000, *size = 142602240 [thread_pthread.c:509]
(MR) addr = 7fffded0, stackaddr = 887fe000, space = 142606640, size = 142602240 [thread_pthread.c:707]
(MR) thid 1: p = 7ffff000, s = 142602240, *addr = 887fe000, *size = 142602240 [thread_pthread.c:509]
(MR) addr = 7fffded0, stackaddr = 887fe000, space = 142606640, size = 142602240 [thread_pthread.c:707]
(MR) thid 1: p = 7ffff000, s = 142602240, *addr = 887fe000, *size = 142602240 [thread_pthread.c:509]
(MR) addr = 7fffe3e4, stackaddr = 887fe000, space = 142605340, size = 142602240 [thread_pthread.c:707]
(MR) thid 7: p = 77771000, s = 524288, *addr = 777f1000, *size = 524288 [thread_pthread.c:509]
(MR) thid 1: p = 7ffff000, s = 142602240, *addr = 887fe000, *size = 142602240 [thread_pthread.c:509]
(MR) addr = 7fffd474, stackaddr = 887fe000, space = 142609292, size = 142602240 [thread_pthread.c:707]

[[e-mail] (Michal Rokos)]

Maybe one more comment: Once the _pthread_stack_info_np() is used, I would recommend to clean up the mess with machine_register_stack_* and usage of rb_ia64_bsp assembly. I guess all the required information is provided within the _pthread_stack_info_t.

[nobu (Nobuyoshi Nakada)]

• Status changed from Feedback to Closed

This issue was solved with changeset r42808.
Michal, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

---

thread_pthread.c: fix compile error on HP-UX

• thread_pthread.c (sys/dyntune.h): for gettune().
• thread_pthread.c (hpux_attr_getstackaddr): fix missing *.
  [ruby-core:56983] [Feature #8793]