Feature #17884


locindex for profiling tools

Added by ko1 (Koichi Sasada) about 1 year ago. Updated about 1 year ago.

Target version:


(MRI internals)

Profiling tools need to record the code location, mainly a pair of file path and line number ("file:line").
To record this pair in 64bit CPU, 8B (VALUE) + 4B (int) = 12B is needed. In general, the number of pairs (file:line) in a interpreter process does not exceed 32bit boundary (4G pairs). st_data_t is 8B (or 4B on 32bit CPU) and we can not store "file:path" information as st_table key/value.

Also getting a line from PC (program counter), is not simple (now we are using succinct bitvector, enough fast and compact data in general, but need some calculations).

To solve the size and the time problem, we introduced new concept "locindex".

"locindex" is unsigned int data structure, maybe 4B in many environments. A "locindex" represents a pair of "iseq" and "PC" (more correctly, "pc_index", given by PC - iseq->body->iseq_encoded).

We can get "locindexL" from "iseqA" with "pcB". "iseqA" will not be freed in this process.
From "locindexL", we can get "iseqA" and "pcB". We can calculate "file:line" information from the iseq/pc pair. "file:line" information is needed to see the profiling results, so the performance of getting "file:line" from a "iseq/pc" pair is not important, in many cases.

"locindex" is calculated by the following pseudo code:

$last_locindex = 1
$global_recorded_ary = []

def locindex iseq, pc # pc is pc_index (nth instruction)
  if iseq->locindex_start == 0 # not recorded yet
    iseq->locindex_start = last_locindex
    $last_locindex += iseq->iseq_size
  iseq->locindex_start + pc

def resolve locindex
    if locindex is in iseq?
      return [iseq, locindx - iseq->locindex_start]

ObjectSpace.trace_object_allocations is one of profiling tool and we can use "locindex" to make it.
I implemented and measure the performance.

The benchmark program:

# This file should be located to the ruby's src directory

require 'objspace/trace'
require 'rdoc/rdoc'
require 'tmpdir'

srcdir = File.expand_path(__dir__)
STDERR.puts srcdir

  dir = File.join(d, 'rdocbench')
  args = %W(--root #{srcdir} --page-dir #{srcdir}/doc --encoding=UTF-8 --no-force-update --all --ri --debug --quiet #{srcdir})
  args << '--op' << dir

  r =
  r.document args


# without 'objspace/trace'
real    0m19.764s
user    0m19.200s
sys     0m0.561s

# with 'objspace/trace'
real    0m42.638s
user    0m41.695s
sys     0m0.920s

# with 'objspace/trace' and locindex
real    0m36.875s
user    0m35.956s
sys     0m0.890s

# with 'objspace/trace' light mode
real    0m27.743s
user    0m26.921s
sys     0m0.820s

Light mode is only recording "locindex".
I believe that most of case it is enough to see the "file:line" pair for performance tuning.

"light-mode" seems more practical.

Actions #1

Updated by ko1 (Koichi Sasada) about 1 year ago

  • Description updated (diff)

Also available in: Atom PDF