Bug #8208: Raise cached exceptions for nonblocking IO to avoid allocation/stack-copying costs

• Status: Rejected
• Assignee: -
• Target version: -
• ruby -v: 2.0.0

[headius (Charles Nutter)]

Currently, all nonblocking IO APIs raise exceptions when the IO channel cannot perform the requested read or write without blocking. This adds a tremendous performance hit to nonblocking IO, since raising an exception requires both allocating the exception object and saving off the backtrace information from the call stack.

Two changes could reduce this cost:

1. Don't provide a full backtrace for these exceptions, or only provide it in a debug or verbose modes. 99% of the rescuers of these exceptions don't ever use the backtrace, so it's wasted overhead.

2. Always raise a cached exception. This completely avoids all allocation costs for nonblocking operations that fail.

JRuby does #1 in JRuby 1.7 series, but does not do #2. I will work on a patch for MRI to do both #1 and #2.

The benefit of this change would be that nonblocking IO has zero wasted overhead. The down side would be that the exceptions raised do not have a useful backtrace unless debug or verbose.

If we did this change, we would not have to introduce a separate API for nonblocking IO that doesn't raise exceptions.

Thoughts?

[headius (Charles Nutter)]

No comments?

Perf change for an implementation of this in JRuby:

Before:

$ jruby -rbenchmark -rsocket -e "s = TCPSocket.new('google.com', 80); 10.times { puts Benchmark.measure { 100_000.times { begin; s.read_nonblock(1000); rescue; end } } }"
3.020000 0.300000 3.320000 ( 1.967000)
0.610000 0.200000 0.810000 ( 0.725000)
0.620000 0.200000 0.820000 ( 0.737000)
0.540000 0.190000 0.730000 ( 0.719000)
0.580000 0.200000 0.780000 ( 0.756000)
0.570000 0.200000 0.770000 ( 0.748000)
0.560000 0.190000 0.750000 ( 0.743000)
0.560000 0.200000 0.760000 ( 0.741000)
0.570000 0.190000 0.760000 ( 0.747000)
0.570000 0.200000 0.770000 ( 0.747000)

After:

$ jruby -rbenchmark -rsocket -e "s = TCPSocket.new('google.com', 80); 10.times { puts Benchmark.measure { 100_000.times { begin; s.read_nonblock(1000); rescue; end } } }"
1.750000 0.230000 1.980000 ( 1.216000)
0.360000 0.180000 0.540000 ( 0.449000)
0.370000 0.190000 0.560000 ( 0.493000)
0.320000 0.180000 0.500000 ( 0.445000)
0.330000 0.170000 0.500000 ( 0.448000)
0.280000 0.170000 0.450000 ( 0.436000)
0.270000 0.160000 0.430000 ( 0.434000)
0.280000 0.170000 0.450000 ( 0.434000)
0.270000 0.160000 0.430000 ( 0.433000)
0.270000 0.170000 0.440000 ( 0.431000)

[normalperson (Eric Wong)]

"headius (Charles Nutter)" [e-mail] wrote:

Perf change for an implementation of this in JRuby:

How does this compare to an implementation which returns symbols
like :wait_*able instead of raising exceptions?

The benefit of this change would be that nonblocking IO has zero
wasted overhead. The down side would be that the exceptions raised do
not have a useful backtrace unless debug or verbose.

Exceptions for nonblocking I/O still makes --debug too noisy.

If we did this change, we would not have to introduce a separate API
for nonblocking IO that doesn't raise exceptions.

I still think it's better to push for an exception-less API for common
errors. EAGAIN/EINPROGRESS/EWOULDBLOCK are far too common (and not
reasonably avoidable) to be considered exceptions.

[headius (Charles Nutter)]

On Apr 16, 2013, at 8:42 PM, Eric Wong [e-mail] wrote:

How does this compare to an implementation which returns symbols
like :wait_*able instead of raising exceptions?

It will be more expensive, but mostly because exception-handling has a fair amount of overhead even with a cached exception object. I suggest this mostly because it may be considered less invasive than adding new APIs.

The benefit of this change would be that nonblocking IO has zero
wasted overhead. The down side would be that the exceptions raised do
not have a useful backtrace unless debug or verbose.

Exceptions for nonblocking I/O still makes --debug too noisy.

Indeed!

I still think it's better to push for an exception-less API for common
errors. EAGAIN/EINPROGRESS/EWOULDBLOCK are far too common (and not
reasonably avoidable) to be considered exceptions.

Agreed…but that hasn't happened yet. I'm hedging my bets :-)

Which issue is tracking an exceptionless nonblocking API? Maybe we can nudge it forward.

• Charlie

[normalperson (Eric Wong)]

Charles Nutter [e-mail] wrote:

Which issue is tracking an exceptionless nonblocking API? Maybe we can
nudge it forward.

https://bugs.ruby-lang.org/issues/5138

Now that CommonRuby exists, it should probably be moved to CommonRuby...

(I try to stay out of API/language design issues, I don't trust myself
in that area)

[headius (Charles Nutter)]

I'm not sure this is really a CommonRuby feature. It would bend some existing behavior (relevant Errno raised by nonblock would no long have a valid trace, except in debug or verbose mode) but other than that behavior remains largely the same.

If I were to summarize the changes:

Impl-specific changes:

• Raise a cached errno exception for each nonblocking event. Any impl could opt not to do this without behavioral difference other than described below.

General changes:

• errno raised for nonblocking events would no longer be required to provide a backtrace, unless verbose or debug.

It might also be possible to use a cached object per-thread but stuff new backtraces into it as a half-way measure, but I think that would largely defeat the gains of this change.

I will have a look at the nonblock API bug and see if I can move it forward.

[headius (Charles Nutter)]

• Status changed from Open to Rejected

https://bugs.ruby-lang.org/issues/5138 has been accepted in a slightly altered form, so I think perhaps this can be rejected. Since it is now possible to use read_nonblock with no exception raise, my proposal is unnecessary.