Feature #6694

Thread.new without block.

Added by Koichi Sasada almost 2 years ago. Updated over 1 year ago.

[ruby-dev:45910]
Status:Assigned
Priority:Normal
Assignee:Koichi Sasada
Category:core
Target version:next minor

Description

=begin
= Abstract

Support Thread.new() without block.

Before: Thread.new(params...){|threadlocalparams| ...}

After: Thread.new(proc: lambda{|tlparams...| ...}, args: params..., otherthread_config...)

= Background

Thread.new creates new Thread object and run passed block in another thread immediately. Thread.new can receive parameters and pass all parameters to block.

Thread.new(a, b, c) do |ta, tb, tc|
# ta, tb, tc is thread local
}

There are some request to specify thread configurations such as stack size described in Ruby 1.9 - Feature #3187. However, we have no way to pass such thread configuration on the Thread.new().

= Proposal

Allow Thread.new() without block. A block will be passed with proc parameter. Passed arguments will be passed with args parameter.

# ex1
Thread.new(){...}
#=>
Thread.new(proc: -> {...})

# ex2
Thread.new(a, b, c){|ta, tb, tc| ...}
#=>
Thread.new(proc: ->(ta, tb, tc){ ... }, params: [a, b, c])

If you want to specify stack size, then:

Thread.new(stack_size: 4096, proc: proc{...}, args: [a, b, c])

Note that I'll make another ticket for thread (and fiber) creation parameters.

This change can be described with the following pseudo code:

def Thread.new(args, &block)
if block
Thread.new_orig(
args, &block)
else
config = args[0] || raise ArgumentError
stacksize = config[:stacksize]
# ... and process another parameters
Thread.new_orig(*config[:args], &config[:proc])
end
end

= Another proposal

On the , Nahi-san proposed that if no block given on Thread.new(), then create "waiting" thread. Thread#run kicks waiting thread with parameters.

th = Thread.new(threadconfigparams)
...
th.run(params){|threadlocalparams|
...
}

We can combine with proc: parameter and this proposal. If Thread.new() doesn't have block and proc: parameter, then making a waiting thread.

NOTE: Because we have already Thread#run, Thread#start is better than Thread#run?

= Note

I don't make any survey on other languages. Please give us your comments.

=end

History

#1 Updated by Koichi Sasada almost 2 years ago

  • Tracker changed from Bug to Feature

#2 Updated by Yusuke Endoh almost 2 years ago

  • Status changed from Open to Assigned

#3 Updated by Brian Shirai almost 2 years ago

I object to this API for at least the two following reasons:

  1. Stack size is an implementation detail and coupling Ruby code to details of a particular implementation is undesirable. Applications may be developed on one implementation and deployed on another. Or details affecting stack size may change between versions of a single implementation.
  2. Stack size may depend on code not in the application or library (eg a library using Thread.new that calls application code or application code that different versions or implementations of a library).

This setting should be a configuration option, not a Ruby method API.

Cheers,
Brian

#4 Updated by Shyouhei Urabe almost 2 years ago

Forgot to mention to @_ko1 that both POSIX and Windows API starts threads immediately when they are created.

It is not that obvious what should happen when a thread that was created is not running. For instance, to join that thread should do what?

#5 Updated by Koichi Sasada almost 2 years ago

(2012/07/20 15:32), brixen (Brian Ford) wrote:

I object to this API for at least the two following reasons:

  1. Stack size is an implementation detail and coupling Ruby code to details of a particular implementation is undesirable. Applications may be developed on one implementation and deployed on another. Or details affecting stack size may change between versions of a single implementation.
  2. Stack size may depend on code not in the application or library (eg a library using Thread.new that calls application code or application code that different versions or implementations of a library).

This setting should be a configuration option, not a Ruby method API.

Maybe you mention to this ticket, don't you?
https://bugs.ruby-lang.org/issues/6695

This ticket only propose thread configuration before thread creation.

--
// SASADA Koichi at atdot dot net

#6 Updated by Koichi Sasada almost 2 years ago

(2012/07/20 15:55), SASADA Koichi wrote:

This ticket only propose thread configuration before thread creation.

Sorry, "configurable thread before running".

--
// SASADA Koichi at atdot dot net

#7 Updated by Koichi Sasada almost 2 years ago

(2012/07/20 15:46), shyouhei (Shyouhei Urabe) wrote:

Forgot to mention to @_ko1 that both POSIX and Windows API starts threads immediately when they are created.

It is not that obvious what should happen when a thread that was created is not running.

On POSIX and Windows threads, the creation API has configuration
parameter. Thread.new doesn't have.

For instance, to join that thread should do what?

Two possible solutions.

  1. raise ThreadError
  2. wait for thread run and exit.

    On second solution, it is easy to understand that the thread created by
    Thread.new without procedure (block or parameter) start and stop
    (Thread#wait) immediately.

    But I prefer 1.

    // SASADA Koichi at atdot dot net

#8 Updated by Shyouhei Urabe almost 2 years ago

ko1 (Koichi Sasada) wrote:

(2012/07/20 15:46), shyouhei (Shyouhei Urabe) wrote:

Forgot to mention to @_ko1 that both POSIX and Windows API starts threads immediately when they are created.

It is not that obvious what should happen when a thread that was created is not running.

On POSIX and Windows threads, the creation API has configuration
parameter. Thread.new doesn't have.

So?

I'm not against adding a new thread constructor.

The problem is that new one do not start.

But I prefer 1.

What happens when Thread.kill called with that thread?
What happens when Thread#backtrace was called?
What if Thread#status?

You have to define all those operations. That is what I call "not obvious".

#9 Updated by Shyouhei Urabe almost 2 years ago

Also, I'd like to let you know that pthreadcreate() comes with their design decisions as RATIONALE.
http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread
create.html

Please read it.

#10 Updated by Brian Shirai almost 2 years ago

ko1 (Koichi Sasada) wrote:

(2012/07/20 15:32), brixen (Brian Ford) wrote:

I object to this API for at least the two following reasons:

  1. Stack size is an implementation detail and coupling Ruby code to details of a particular implementation is undesirable. Applications may be developed on one implementation and deployed on another. Or details affecting stack size may change between versions of a single implementation.
  2. Stack size may depend on code not in the application or library (eg a library using Thread.new that calls application code or application code that different versions or implementations of a library).

This setting should be a configuration option, not a Ruby method API.

Maybe you mention to this ticket, don't you?
https://bugs.ruby-lang.org/issues/6695

This ticket only propose thread configuration before thread creation.

I don't think I understand your question. To summarize my objection to this proposed API change: Thread stack size should be something set at the VM level completely outside of Ruby code. Ruby code should not be coupled with implementation details.

Cheers,
Brian

#11 Updated by Koichi Sasada almost 2 years ago

(2012/07/20 22:42), brixen (Brian Ford) wrote:

I don't think I understand your question. To summarize my objection to this proposed API change: Thread stack size should be something set at the VM level completely outside of Ruby code. Ruby code should not be coupled with implementation details.

I think I understand your question. And your question should be
discussed on ticket 6695 https://bugs.ruby-lang.org/issues/6695.

My proposal is not adding "initial stack size parameter for thread
creation".
It is only "configurable thread parameters". More general framework.

I propose "stack size parameter" at next ticket (using this ticket):
https://bugs.ruby-lang.org/issues/6695

This ticket (6694):
Change Thread.new() API to accept any initial parameters.

Next ticket (6695):
New parameters including stack size (machine stack size and VM
stack size).
Not only stack size, but also "name" and others.
I think "name" is killer usage of this ticket (6694).

--
// SASADA Koichi at atdot dot net

#12 Updated by Brian Shirai almost 2 years ago

ko1 (Koichi Sasada) wrote:

(2012/07/20 22:42), brixen (Brian Ford) wrote:

I don't think I understand your question. To summarize my objection to this proposed API change: Thread stack size should be something set at the VM level completely outside of Ruby code. Ruby code should not be coupled with implementation details.

I think I understand your question. And your question should be
discussed on ticket 6695 https://bugs.ruby-lang.org/issues/6695.

My proposal is not adding "initial stack size parameter for thread
creation".
It is only "configurable thread parameters". More general framework.

I propose "stack size parameter" at next ticket (using this ticket):
https://bugs.ruby-lang.org/issues/6695

This ticket (6694):
Change Thread.new() API to accept any initial parameters.

Next ticket (6695):
New parameters including stack size (machine stack size and VM
stack size).
Not only stack size, but also "name" and others.
I think "name" is killer usage of this ticket (6694).

Thread#name doesn't require a new API for Thread.new. Thread#name makes sense as something that can be set any time.

The pthread_create() design rationale referenced by Urabe-san is relevant to this proposed API. For the same reasons discussed there, I would object to this API change. I don't see any real need for this API if stack size is not one of the things being set. Hence, this API proposal really looks like just a thin rationale for that proposal.

Cheers,
Brian

#13 Updated by Charles Nutter almost 2 years ago

I will comment on stack sizing on the other issue.

As far as passing params and not starting the thread, it doesn't seem like a bad API. There are other thread characteristics that could be passed at construction time like abortonexception, priority, and potentially things like "start" (to autostart as the current API), "daemon" to create threads that will keep the VM alive after main thread exits, and so on.

FWIW, JVM threads do not run until explicitly started. The start of the actual native thread is deferred until Thread#start is called. Thread-runtime operations raise exceptions if called before the thread is started. Simple enough.

#14 Updated by Motohiro KOSAKI almost 2 years ago

I don't think I understand your question. To summarize my objection to this proposed API change: Thread stack size should be something set at the VM level completely outside of Ruby code. Ruby code should not be coupled with implementation details.

If I understand correctly, you suggested Thread and Fiber should have
enough large stack size. I 100% agree. Then almost all ruby
programmers don't need bother stack size issue.

The problem is, current fiber implementation have merely very small
stack size and it often caused stack overflow issue. Why it is so
small? Because of, larger stack size restricted maximum number of
fibers. Think, stack-size * number-of-fibers should be < 3G if system
is running on 32bit OS.

So, we hope to increase stack size for just your opinion. but it has
one down size. it reduce a number of maximum fibers. The next question
is, anybody need such so many thread/fibers? Is this real issue?

Unfortunately, we can't answer it. It depend on ruby scripts. Out of
ruby world, I know some game programmer prefer to create a lot of
fibers. But I don't know real ruby use case. However, I hope to keep
a way to create a lot of fibers if programmer don't hesitate deep
system/cpu depending scripts.

Finally, if you agree fiber issue and you think only fiber should have
stack size parameter, can you please explain why do you dislike thread
and fiber have the same interface? Usually, thread and fiber should
keep same or similar interface because it reduce learning cost.

Thank you.

#15 Updated by Koichi Sasada over 1 year ago

=begin
Some developers proposed that other method names like: Thread.conf_new should be introduced to specify per thread parameters.

Because Thread.new(...) without block causes miss like:

Thread.new(name: 'foo'){
}

In this case, {name: 'foo'} will be passed as block parameter for thread and it doesn't make any warnings.

I'm not sure the name 'Thread.confnew()' is good name or not.
Thread.new
configured(...) is too long?

Thread.new2 is good joke.

=end

#16 Updated by Koichi Sasada over 1 year ago

  • Target version changed from 2.0.0 to next minor

I don't have good name about it.
I postpone this ticket.

Also available in: Atom PDF