Ruby » ruby-trunk

Hi,

At Wed, 28 Oct 2009 02:03:01 +0900,
Suraj Kurapati wrote in [ruby-core:26361]:
> I am attaching a "ruby_bind_stack.patch" patch file 
> that adds a ruby_bind_stack() function to the Ruby C API.

This patch does not work with multithreading at all.

-- 
Nobu Nakada

I'm embedding a Ruby 1.9.1 in my app, and it would die with segmentation fault, and I was suspicious about the stack, as it's multithreaded. I applied your patch and it looks fine so far. I'm using the following code (and assume that stack grows upward):

static void pthread_get_stack(void **stack_begin, void **stack_end) {
	size_t stack_size;
#if defined(HAVE_STACKADDR_NP) && defined(HAVE_GET_STACKSIZE_NP) /* MacOS X */
	pthread_t t_id = pthread_self();
	*stack_begin = pthread_get_stackaddr_np(t_id);
	stack_size = pthread_get_stacksize_np(t_id);
#else /* Linux */
	pthread_attr_t attr;
	pthread_getattr_np(pthread_self(), &attr);
	pthread_attr_getstack(&attr, stack_begin, &stack_size);
#endif
	*stack_end = *stack_begin + stack_size;
}

Hi Roman,

I did not understand your code example.
Where do you call ruby_bind_stack() ?

Without that call, I don't see how my patch
can make any difference to your program.

Thanks.

Sorry for the lack of explanation, I thought that it was implicit and apparent.
After I get thread's stack_begin and stack_end I'm calling your bind stack function, of course.

Roman, thanks for clarifying.

Hi,

According to Matz's suggestion in [ruby-core:25139], I wrote
a detailed explanation of the problem this patch solves. I 
hope this explanation is helpful.  Please do not hesitate
to ask for clarifications or to correct any misunderstandings.

Thanks for your thoughtful consideration.

== Introduction

The patch adds a ruby_bind_stack() function to the Ruby C API.
This function allows the person who is embedding Ruby to
tell the Ruby GC about the stack boundaries of the embedded
environment:

  void ruby_bind_stack(VALUE *lower_bound, VALUE *upper_bound);

In order to understand why this function is important, please
consider the following two modes of operation: normal & embedded.


== Normal operation: Ruby runs in a C program's main()

Initially, Ruby assumes that the stack of Ruby's main
thread exists in a high memory address range, like this:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  (low memory address)

As Ruby runs, the lower boundary is adjusted (by the
SET_STACK_END macro) to reflect the machine stack pointer:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary (not changed)

  0xc0ff1e80    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  (low memory address)


== Embedded operation: Ruby runs inside a C coroutine

Initially, Ruby assumes that the stack of Ruby's main
thread exists in a high memory address range, like this:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  (low memory address)

However, the stack of the C coroutine (which runs Ruby)
exists at a low memory address range, because it is
statically allocated:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  0x086032a0    System V context's stack upper boundary

  0x082032a0    System V context's stack lower boundary

  (low memory address)

As Ruby runs, the lower boundary is adjusted (by the
SET_STACK_END macro) to reflect the machine stack pointer:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0x086032a0    System V context's stack upper boundary

  0x08601680    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  0x082032a0    System V context's stack lower boundary

  (low memory address)


See the problem?  Ruby's stack and the C coroutine stack
do not agree.  They overlap!

This situation becomes worse (and causes a segfault) when
the Ruby GC runs: it marks VALUEs in the Ruby stack, which
currently contains all of the heap memory!  Somewhere in
the vast heap memory, it finds and dereferences a NULL value
and BOOM! a segfault occurs.  :-)

To solve this problem, the ruby_bind_stack() function corrects
Ruby's stack to reflect the stack boundaries of the C coroutine:

  (high memory address)

  0x086032a0    Ruby's stack upper boundary and also
                System V context's stack upper boundary

  0x08601680    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  0x082032a0    System V context's stack lower boundary

  (low memory address)

Now, when the Ruby GC runs, it marks VALUEs in the correct
memory region.  It does not travel into heap memory and
cause a segfault.

That is all.  Thanks for reading!

Hi,

I would like to say that without applying this patch, my ruby interpreter, embedded in a pthread, would cause a segmentation fault as soon as GC was invoked. I would like to see this applied to 1.9.1 as well as http://redmine.ruby-lang.org/issues/show/2279 . Without these, it's hardly possible to have ruby 1.9.1 embedded in a useful way.

Thanks,
--Roman 


----- Original Message ----
From: Suraj Kurapati <redmine@ruby-lang.org>
To: ruby-core@ruby-lang.org
Sent: Thu, November 5, 2009 9:27:17 AM
Subject: [ruby-core:26550] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine

Issue #2294 has been updated by Suraj Kurapati.


Hi,

According to Matz's suggestion in [ruby-core:25139], I wrote
a detailed explanation of the problem this patch solves. I 
hope this explanation is helpful.  Please do not hesitate
to ask for clarifications or to correct any misunderstandings.

Thanks for your thoughtful consideration.

== Introduction

The patch adds a ruby_bind_stack() function to the Ruby C API.
This function allows the person who is embedding Ruby to
tell the Ruby GC about the stack boundaries of the embedded
environment:

  void ruby_bind_stack(VALUE *lower_bound, VALUE *upper_bound);

In order to understand why this function is important, please
consider the following two modes of operation: normal & embedded.


== Normal operation: Ruby runs in a C program's main()

Initially, Ruby assumes that the stack of Ruby's main
thread exists in a high memory address range, like this:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  (low memory address)

As Ruby runs, the lower boundary is adjusted (by the
SET_STACK_END macro) to reflect the machine stack pointer:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary (not changed)

  0xc0ff1e80    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  (low memory address)


== Embedded operation: Ruby runs inside a C coroutine

Initially, Ruby assumes that the stack of Ruby's main
thread exists in a high memory address range, like this:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  (low memory address)

However, the stack of the C coroutine (which runs Ruby)
exists at a low memory address range, because it is
statically allocated:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0xbffff1f0    Ruby's stack lower boundary

  0x086032a0    System V context's stack upper boundary

  0x082032a0    System V context's stack lower boundary

  (low memory address)

As Ruby runs, the lower boundary is adjusted (by the
SET_STACK_END macro) to reflect the machine stack pointer:

  (high memory address)

  0xc1bff1f0    Ruby's stack upper boundary

  0x086032a0    System V context's stack upper boundary

  0x08601680    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  0x082032a0    System V context's stack lower boundary

  (low memory address)


See the problem?  Ruby's stack and the C coroutine stack
do not agree.  They overlap!

This situation becomes worse (and causes a segfault) when
the Ruby GC runs: it marks VALUEs in the Ruby stack, which
currently contains all of the heap memory!  Somewhere in
the vast heap memory, it finds and dereferences a NULL value
and BOOM! a segfault occurs.  :-)

To solve this problem, the ruby_bind_stack() function corrects
Ruby's stack to reflect the stack boundaries of the C coroutine:

  (high memory address)

  0x086032a0    Ruby's stack upper boundary and also
                System V context's stack upper boundary

  0x08601680    Ruby's stack lower boundary
                (after update by SET_STACK_END)

  0x082032a0    System V context's stack lower boundary

  (low memory address)

Now, when the Ruby GC runs, it marks VALUEs in the correct
memory region.  It does not travel into heap memory and
cause a segfault.

That is all.  Thanks for reading!
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2294

----------------------------------------
http://redmine.ruby-lang.org

Hi,

Sorry to be impatient, but has there been any
further decision or consideration about this patch?

The only feedback I've received so far is that:

* An early version of this patch did not support
  multi-threading (thanks to Mr. Nobu).

* A later version of this patch worked for embedding
  Ruby 1.9 inside a pthread (thanks to Mr. Roman).

The silent suspense is "killing" me, so to speak. :-)

Thanks for your consideration.

Hi,

At Wed, 18 Nov 2009 15:55:07 +0900,
Suraj Kurapati wrote in [ruby-core:26797]:
> Sorry to be impatient, but has there been any
> further decision or consideration about this patch?

Sorry to be late.

> The only feedback I've received so far is that:
> 
> * An early version of this patch did not support
>   multi-threading (thanks to Mr. Nobu).
> 
> * A later version of this patch worked for embedding
>   Ruby 1.9 inside a pthread (thanks to Mr. Roman).

Switching stack using setcontext() can't work on all platforms.
For instance, on NetBSD and older LinuxThread stack address is
tightly bound to thread, and can't be changed.  That is, your
strategy is not portable.

Why don't you simply use a thread instead?

-- 
Nobu Nakada

Hi,

In message "Re: [ruby-core:26803] [Feature #2294](Rejected) [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
    on Wed, 18 Nov 2009 18:42:31 +0900, Nobuyoshi Nakada <redmine@ruby-lang.org> writes:

|I don't think it's good idea to it as public API.

Hmm, but you still should not ignore the fact described in
[ruby-core:26661].  If the patch solve a serious problem under one
condition (SEGV on embedding environment), you cannot reject it just
saying 'not a good idea'.

							matz.

Hi,

Nobu Nakada wrote:
> Switching stack using setcontext() can't work on all platforms.
> For instance, on NetBSD and older LinuxThread stack address is
> tightly bound to thread, and can't be changed.  That is, your
> strategy is not portable.

You are referring only to my System V context example, right?  If so,
please note that I also provided a second example that uses libpcl[1]
which "can use either the ucontext.h functionalities... or the standard
longjmp()/setjmp()" and "is easily portable on almost every Unix system
and on Windows" [1].

I will create a thrid example that uses libpthread to demonstrate
how this patch lets you embed Ruby 1.9 inside a pthread.  (Note that this
patch has already allowed Mr. Roman to embed Ruby 1.9 inside a pthread.)

> Why don't you simply use a thread instead?

Do you mean embedding Ruby inside a pthread?

Thanks for your consideration.

[1]: http://www.xmailserver.org/libpcl.html

Hi,

In message "Re: [ruby-core:26816] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
    on Thu, 19 Nov 2009 04:30:49 +0900, Suraj Kurapati <redmine@ruby-lang.org> writes:

|You are referring only to my System V context example, right?  If so,
|please note that I also provided a second example that uses libpcl[1]
|which "can use either the ucontext.h functionalities... or the standard
|longjmp()/setjmp()" and "is easily portable on almost every Unix system
|and on Windows" [1].

As far as I understand, libpcl is under GPL, that cannot be used in
the core Ruby.  Since Ruby is not covered by GPL only.

							matz.

Hi,

matz writes:
> Suraj Kurapati writes:
> |please note that I also provided a second example that uses libpcl[1]
> 
> As far as I understand, libpcl is under GPL, that cannot be used in
> the core Ruby.  Since Ruby is not covered by GPL only.

My patch simply adds a ruby_bind_stack() method to the Ruby C API:

  ruby_bind_stack_r25604.patch      (attached to Feature #2294)

It does *not* use ucontext, libpcl, pthreads or any other coroutine
libraries.  These libraries are only used in the example test cases
I provided to demonstrate how the ruby_bind_stack() function can be
used to embed Ruby inside a coroutine environment:

  ruby-libpcl-dynamic-stack.tgz     (attached to Feature #2294)
  ruby-libpcl-static-stack.tgz      (attached to Feature #2294)
  ruby-ucontext-static-stack.tgz    (attached to Feature #2294)
  ruby-ucontext-dynamic-stack.tgz   (attached to Feature #2294)

So there is no problem about license compatibility, right?

Thanks for your consideration.

Hi,

In message "Re: [ruby-core:26818] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
    on Thu, 19 Nov 2009 06:15:50 +0900, Suraj Kurapati <redmine@ruby-lang.org> writes:

|> As far as I understand, libpcl is under GPL, that cannot be used in
|> the core Ruby.  Since Ruby is not covered by GPL only.
|
|My patch simply adds a ruby_bind_stack() method to the Ruby C API:
|
|  ruby_bind_stack_r25604.patch      (attached to Feature #2294)

I am sorry about my misunderstanding.  In that case, there should not
be any license issue.  I'd wait Nobu to express his opinion.

							matz.

Hi,

This issue is marked as "Rejected", but it seems that Matz agreed
that this issue still needs further consideration, so please reset
this issue's status to "Open".

Thanks.

Encountered this problem while embedding Ruby in a pthread'ed plug-in. I applied the patch against r25604 but unfortunately it did not solve the problem. When pthread_main_np() == 0 it crashes. When pthread_main_np() != 0 it works.

I will be ahppy to help test this and provide more information if needed. Thanks.

Hi Ammar,

That is an interesting discovery, because the patch only corrects the
stack boundaries of Ruby's main thread (when pthread_main_np() == 1).

It deliberately ignores non-main threads when doing the correction 
because non-main threads are also pthreads and AFAIK they allocate
and manage their own stacks on the heap.

If possible, please modify the example-pthread-*.tgz examples to
demonstrate the failure you described and attach them to this issue.

Thanks for your consideration.

Hi Suraj,

Perhaps I wishfully believed your patch to be the needed solution, but what I'm seeing is definitely occurring when pthread_main_np() == 0.

With 1.9.1-p243 I was seeing random problems that seemed stack related. After searching through the reported issues, I decided to try r25604 and that's when the error became exactly, and consistently, what you described in #2258. That's how I found this patch.

After looking at your examples I don't think they mirror my situation and would need more than a modification to make them do so. Unfortunately I will not have the time for building a test case that mimics my situation accurately before another week or two. 

I'm not sure this makes any difference really, but in my case Ruby is embedded inside a dynamically loadable plug-in. The target host program comes in two editions; a client that runs its plug-ins from the main thread, and a server that runs them in a child thread. The client works fine while the server crashes on the first call to require. To create an accurate test I think I need to reproduce this exact situation. I'm not sure though, I have to investigate this when time allows.

Thank you for the hard work and your consideration.

Hi Ammar,

Please try r25842 or newer (with and without my patch) and see if it solves
your problem.  That particular revision solves the "[BUG] object allocation
during garbage collection phase" error (reported in #2258) you encountered.

Hopefully, yours will turn out to be an unrelated issue so I can make 
forward progress on getting this patch accepted (someday!!).  :-)

Thanks for your consideration.

Hi Nobu,

I combined the various coroutine library examples into a single one:

  http://github.com/sunaku/ruby-coroutine-example

You can run the example like this:

  # libpcl
  sh run.sh pcl static path_to_your_ruby_svn_trunk_installation
  sh run.sh pcl dynamic path_to_your_ruby_svn_trunk_installation

  # POSIX threads
  sh run.sh pthread static path_to_your_ruby_svn_trunk_installation
  sh run.sh pthread dynamic path_to_your_ruby_svn_trunk_installation

  # System V contexts
  sh run.sh ucontext static path_to_your_ruby_svn_trunk_installation
  sh run.sh ucontext dynamic path_to_your_ruby_svn_trunk_installation

I tried this on ruby 1.9.3dev (2010-05-25 trunk 28007) [i686-linux]
with my ruby_bind_stack patch applied and observed that the patch is
still necessary to make this example work.

You can actually test multiple Ruby versions by passing:

  sh run.sh ... ~/.multiruby/install/*

Please retry this example when you have a chance.

Thanks for your consideration.

Hi,

Could we finally get this patch commited, please? It's not like it's a thousand line behemoth and it solves a very real problem - it's impossible to embed Ruby into a pthread without it. I really see no reason not to commit this.

Hi,

Ko1, could you respond to this issue, please?  Either positively or
negatively, we should not leave this untouched.

							matz.

In message "Re: [ruby-core:33727] [Ruby 1.9-Feature#2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
    on Thu, 16 Dec 2010 00:27:40 +0900, Anonymous Anonymous <redmine@ruby-lang.org> writes:

|Could we finally get this patch commited, please? It's not like it's a thousand line behemoth and it solves a very real problem - it's impossible to embed Ruby into a pthread without it. I really see no reason not to commit this.
|http://redmine.ruby-lang.org/issues/show/2294

Hi,

Suraj, I'm sorry for late response.  I missed this thread.

I read the last patch:
http://redmine.ruby-lang.org/attachments/download/1153

and I need to say "no".

As nobu said at first, this patch is not considering the multi-threading.
(and using global variables should not be accepted :)  The patch is too
ad-hoc modification)


I propose another API.

idea 1:

// API for C extension.
// User needs to know thread value.
rb_thread_set_stack(VALUE thread_val, upper, lower) {
  th = thread_data(thread_val);
  th->upper = upper;
  th->lower = lower;
}

idea 2:

// API called from not a Ruby world
ruby_bind_stack_for_current_native_thread(upper, lower) {
  th = thread_data_for_current_native_thread();

  if (th == 0) {
    // Current native thread does not have
    // the related ruby thread.
    return 0;
  }

  th->upper = upper;
  th->lower = lower;
  return 1;
}


BTW, how to get the correct "upper"/"lower" address of stack?


(2010/12/16 0:37), Yukihiro Matsumoto wrote:
> Hi,
> 
> Ko1, could you respond to this issue, please?  Either positively or
> negatively, we should not leave this untouched.
> 
> 							matz.
> 
> In message "Re: [ruby-core:33727] [Ruby 1.9-Feature#2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
>     on Thu, 16 Dec 2010 00:27:40 +0900, Anonymous Anonymous <redmine@ruby-lang.org> writes:
> 
> |Could we finally get this patch commited, please? It's not like it's a thousand line behemoth and it solves a very real problem - it's impossible to embed Ruby into a pthread without it. I really see no reason not to commit this.
> |http://redmine.ruby-lang.org/issues/show/2294
> 


-- 
// SASADA Koichi at atdot dot net

Hi,

In message "Re: [ruby-core:33730] Re: [Ruby 1.9-Feature#2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine"
    on Thu, 16 Dec 2010 02:48:07 +0900, SASADA Koichi <ko1@atdot.net> writes:
|
|Hi,
|
|Suraj, I'm sorry for late response.  I missed this thread.
|
|I read the last patch:
|http://redmine.ruby-lang.org/attachments/download/1153
|
|and I need to say "no".
|
|As nobu said at first, this patch is not considering the multi-threading.
|(and using global variables should not be accepted :)  The patch is too
|ad-hoc modification)
|
|
|I propose another API.
|
|idea 1:
|
|// API for C extension.
|// User needs to know thread value.
|rb_thread_set_stack(VALUE thread_val, upper, lower) {
|  th = thread_data(thread_val);
|  th->upper = upper;
|  th->lower = lower;
|}
|
|idea 2:
|
|// API called from not a Ruby world
|ruby_bind_stack_for_current_native_thread(upper, lower) {
|  th = thread_data_for_current_native_thread();
|
|  if (th == 0) {
|    // Current native thread does not have
|    // the related ruby thread.
|    return 0;
|  }
|
|  th->upper = upper;
|  th->lower = lower;
|  return 1;
|}
|
|
|BTW, how to get the correct "upper"/"lower" address of stack?

I am afraid that there's no portable and/or reliable way.
Conservative GC does similar thing, maybe you can steal boundary
values from it.

							matz.

Hi,

SASADA Koichi wrote in post #968635:
> I read the last patch:
> http://redmine.ruby-lang.org/attachments/download/1153
>
> and I need to say "no".
>
> As nobu said at first, this patch is not considering the
> multi-threading.
> (and using global variables should not be accepted :)  The patch is too
> ad-hoc modification)

Thanks for your feedback!  I must confess that I did not really
understand how my patch did not support multi-threading, but after
reading your proposed API, I finally understand what Nobu was talking
about. :)

I agree with your feeling and I would like to follow your proposed API.

Thanks for your consideration.

-- 
Posted via http://www.ruby-forum.com/.

(2010/12/16 14:23), Suraj Kurapati wrote:
> Thanks for your feedback!  I must confess that I did not really
> understand how my patch did not support multi-threading, but after
> reading your proposed API, I finally understand what Nobu was talking
> about. :)
> 
> I agree with your feeling and I would like to follow your proposed API.

Could you give me a concrete example? (Execution flow)
(I'm sorry if I missed the example you already posted)

-- 
// SASADA Koichi at atdot dot net

SASADA Koichi wrote in post #968830:
> Could you give me a concrete example? (Execution flow)

Please see http://www.ruby-forum.com/topic/198119#866383

> (I'm sorry if I missed the example you already posted)

I also posted an example demonstration here:
http://www.ruby-forum.com/topic/198119#914657

Thanks for your consideration.

-- 
Posted via http://www.ruby-forum.com/.