Project

General

Profile

Actions
Copy link

Feature #20861

open

Add an environment variable for tuning the default thread quantum

Added by tenderlovemaking (Aaron Patterson) 17 days ago. Updated 4 days ago.

Status:
Open
Assignee:
-
Target version:
-
[ruby-core:119683]

Description

The default thread quantum is currently hard coded at 100ms. This can impact multithreaded systems that are trying to process Ruby level CPU bound work at the same time as IO work.

I would like to add an environment variable RUBY_THREAD_DEFAULT_QUANTUM_MS that allows users to specify the default thread quantum (in milliseconds) via an environment variable. It defaults to our current default of 100ms. I've submitted the patch here.

Here is a Ruby program to demonstrate the problem:

def measure
  x = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  yield
  Process.clock_gettime(Process::CLOCK_MONOTONIC) - x
end

def fib(n)
  if n < 2
    n
  else
    fib(n-2) + fib(n-1)
  end
end

# find fib that takes ~500ms
fib_i = 50.times.find { |i| measure { fib(i) } >= 0.05 }
sleep_i = measure { fib(fib_i) }

threads = [
  Thread.new {
    100.times {
      sleep(sleep_i)
      # sometimes stalled waiting for fib's quantum to finish
    }
    puts "done 1"
  },
  Thread.new { 100.times { fib(fib_i) }; puts "done 2" },
]

# We expect the total time to be about 100 * sleep_i (~5 seconds) because
# theoretically the sleep thread could be done nearly completely in parallel to
# the fib thread.
#
# But because the `sleep` thread is iterating over the sleep call, it must wait
# for the `fib` thread to complete its quantum, before it can start the next iteration.
#
# This means each sleep iteration could take up to `sleep_i + 100ms`
#
# We're calling that stalled time "waste"
total = measure { threads.each(&:join) }
waste = total - (sleep_i * 100)
p TOTAL: total, WASTE: waste

The program has two threads. One thread is using CPU time by computing fib in a loop. The other thread is simulating IO time by calling sleep in a loop. When the sleep call completes, it can stall, waiting for the quantum in the fib thread to expire. That means that each iteration on sleep can actually take sleep time + thread quantum, or in this case ~600ms when we expected it to only take ~500ms.

Ideally, the above program would take 500ms * 100 since all sleep calls should be able to execute in parallel with the fib calls. Of course this isn't true because the sleep thread must acquire the GVL before it can continue the next iteration, so there will always be some overhead. This feature is for allowing people to tune that overhead.

If we run this program with the default quantum the output looks like this:

$ ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T14:49:50Z quantum-computing c7708d22c3) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 12.672821999993175, WASTE: 4.960721996147186}

The output shows that our program spent about 5 seconds stalled, waiting to acquire the GVL.

With this patch we can lower the default quantum, and the output is like this:

$ RUBY_THREAD_DEFAULT_QUANTUM_MS=10 ./miniruby -v fibtest.rb
ruby 3.4.0dev (2024-11-01T22:06:35Z quantum-computing 087500643d) +PRISM [arm64-darwin24]
done 2
done 1
{TOTAL: 8.898526000091806, WASTE: 1.4168260043952614}

Specifying the ENV to change the quantum to 10ms lowered our waste in the program to ~1.4 seconds.

It's common for web applications to do mixed CPU and IO bound tasks in threads (see the Puma webserver), so it would be great if there was a way to customize the thread quantum depending on your application's workload.

Actions
Copy link

Also available in: Atom PDF

Like3
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0