Bug #14357: thread_safe tests suite segfaults

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
• Backport: 2.3: REQUIRED, 2.4: DONE, 2.5: DONE

[vo.x (Vit Ondruch)]

The thread_safe gem is not maintained anymore, but I don't see any reason why its test suite should segfault with Ruby 2.5.

+ rspec -Ilib spec
Randomized with seed 62008
............./builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/support/threadsafe_test.rb:44: [BUG] Segmentation fault at 0x000000000000000c
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
-- Control frame information -----------------------------------------------
c:0011 p:0009 s:0073 e:000070 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/support/threadsafe_test.rb:44 [FINISH]
c:0010 p:0008 s:0066 e:000065 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/non_concurrent_cache_backend.rb:1
c:0009 p:0006 s:0061 e:000060 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/mri_cache_backend.rb:17
c:0008 p:0011 s:0055 e:000054 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/cache.rb:72
c:0007 p:0033 s:0047 e:000046 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:411
c:0006 p:0058 s:0036 e:000035 BLOCK  /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:441 [FINISH]
c:0005 p:---- s:0031 e:000030 CFUNC  :times
c:0004 p:0020 s:0027 e:000026 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:435
c:0003 p:0082 s:0017 e:000016 METHOD /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:397
c:0002 p:0025 s:0006 e:000005 BLOCK  /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:378 [FINISH]
c:0001 p:---- s:0003 e:000002 (none) [FINISH]
-- Ruby level backtrace information ----------------------------------------
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:378:in `block (2 levels) in run_thread_loop'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:397:in `setup_sync_and_start_loop'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:435:in `_put_if_absent_loop_outer_multiple_keys'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:435:in `times'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:441:in `block in _put_if_absent_loop_outer_multiple_keys'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/thread_safe/cache_loops_spec.rb:411:in `_put_if_absent_loop_inner_multiple_keys'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/cache.rb:72:in `put_if_absent'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/mri_cache_backend.rb:17:in `compute_if_absent'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/non_concurrent_cache_backend.rb:12:in `[]'
/builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/support/threadsafe_test.rb:44:in `eql?'
-- Machine register context ------------------------------------------------
 RIP: 0x00007f7c631f030f RBP: 0x00005577af14e110 RSP: 0x00007f7c5acea380
 RAX: 0x0000000000000000 RBX: 0x00007f7c634eae00 RCX: 0x00007f7c5aced258
 RDX: 0x0000000000000004 RDI: 0x00005577af09a890 RSI: 0x00005577aecc5bd8
  R8: 0x00005577aecc5bb0  R9: 0x0000000000000000 R10: 0x0000000000000000
 R11: 0x0000000000000000 R12: 0x00005577aecc5bd8 R13: 0x00005577af09a890
 R14: 0x00005577af14e128 R15: 0x00007f7c5adece00 EFL: 0x0000000000010293
-- C level backtrace information -------------------------------------------
/lib64/libruby.so.2.5(0x7f7c6320c5f5) [0x7f7c6320c5f5]
/lib64/libruby.so.2.5(0x7f7c6320c82c) [0x7f7c6320c82c]
/lib64/libruby.so.2.5(0x7f7c630d7b34) [0x7f7c630d7b34]
/lib64/libruby.so.2.5(0x7f7c6319cfe2) [0x7f7c6319cfe2]
/lib64/libpthread.so.0(0x7f7c62e3a160) [0x7f7c62e3a160]
/lib64/libruby.so.2.5(0x7f7c631f030f) [0x7f7c631f030f]
/lib64/libruby.so.2.5(0x7f7c631fa208) [0x7f7c631fa208]
/lib64/libruby.so.2.5(0x7f7c631ff674) [0x7f7c631ff674]
/lib64/libruby.so.2.5(0x7f7c63203cad) [0x7f7c63203cad]
/lib64/libruby.so.2.5(0x7f7c63204d9a) [0x7f7c63204d9a]
/lib64/libruby.so.2.5(rb_funcallv+0x2e) [0x7f7c6320557e]
/lib64/libruby.so.2.5(rb_eql+0x6c) [0x7f7c63134adc]
/lib64/libruby.so.2.5(0x7f7c630f71a1) [0x7f7c630f71a1]
/lib64/libruby.so.2.5(0x7f7c631a574a) [0x7f7c631a574a]
/lib64/libruby.so.2.5(st_lookup+0x4b) [0x7f7c631a64cb]
/lib64/libruby.so.2.5(rb_hash_aref+0x1f) [0x7f7c630f9ddf]
/lib64/libruby.so.2.5(0x7f7c631f89d2) [0x7f7c631f89d2]
/lib64/libruby.so.2.5(0x7f7c631ff674) [0x7f7c631ff674]
/lib64/libruby.so.2.5(0x7f7c63201234) [0x7f7c63201234]
/lib64/libruby.so.2.5(0x7f7c631283b0) [0x7f7c631283b0]
/lib64/libruby.so.2.5(0x7f7c631f4e99) [0x7f7c631f4e99]
/lib64/libruby.so.2.5(0x7f7c631f9815) [0x7f7c631f9815]
/lib64/libruby.so.2.5(0x7f7c631ff674) [0x7f7c631ff674]
/lib64/libruby.so.2.5(0x7f7c63202860) [0x7f7c63202860]
/lib64/libruby.so.2.5(0x7f7c6320299c) [0x7f7c6320299c]
/lib64/libruby.so.2.5(0x7f7c631cda51) [0x7f7c631cda51]
/lib64/libruby.so.2.5(0x7f7c631cdeb3) [0x7f7c631cdeb3]
/lib64/libpthread.so.0(0x75e1) [0x7f7c62e2f5e1]
/lib64/libc.so.6(clone+0x3f) [0x7f7c623950ef]
-- Other runtime information -----------------------------------------------
* Loaded script: /usr/bin/rspec
* Loaded features:
    0 enumerator.so
    1 thread.rb
    2 rational.so
    3 complex.so
    4 /usr/lib64/ruby/enc/encdb.so
    5 /usr/lib64/ruby/enc/trans/transdb.so
    6 /usr/lib64/ruby/rbconfig.rb
    7 /usr/share/rubygems/rubygems/compatibility.rb
    8 /usr/share/rubygems/rubygems/defaults.rb
    9 /usr/share/rubygems/rubygems/deprecate.rb
   10 /usr/share/rubygems/rubygems/errors.rb
   11 /usr/share/rubygems/rubygems/version.rb
   12 /usr/share/rubygems/rubygems/requirement.rb
   13 /usr/share/rubygems/rubygems/platform.rb
   14 /usr/share/rubygems/rubygems/basic_specification.rb
   15 /usr/share/rubygems/rubygems/stub_specification.rb
   16 /usr/share/rubygems/rubygems/util/list.rb
   17 /usr/lib64/ruby/stringio.so
   18 /usr/share/rubygems/rubygems/specification.rb
   19 /usr/share/rubygems/rubygems/exceptions.rb
   20 /usr/share/rubygems/rubygems/defaults/operating_system.rb
   21 /usr/share/rubygems/rubygems/dependency.rb
   22 /usr/share/rubygems/rubygems/core_ext/kernel_gem.rb
   23 /usr/share/ruby/monitor.rb
   24 /usr/share/rubygems/rubygems/core_ext/kernel_require.rb
   25 /usr/share/rubygems/rubygems.rb
   26 /usr/share/rubygems/rubygems/path_support.rb
   27 /usr/share/rubygems/rubygems/bundler_version_finder.rb
   28 /usr/share/ruby/tsort.rb
   29 /usr/share/rubygems/rubygems/request_set/gem_dependency_api.rb
   30 /usr/share/rubygems/rubygems/request_set/lockfile/parser.rb
   31 /usr/share/rubygems/rubygems/request_set/lockfile/tokenizer.rb
   32 /usr/share/rubygems/rubygems/request_set/lockfile.rb
   33 /usr/share/rubygems/rubygems/request_set.rb
   34 /usr/share/rubygems/rubygems/util.rb
   35 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/gem_metadata.rb
   36 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/errors.rb
   37 /usr/share/ruby/set.rb
   38 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/action.rb
   39 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/add_edge_no_circular.rb
   40 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/add_vertex.rb
   41 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/delete_edge.rb
   42 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/detach_vertex_named.rb
   43 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/set_payload.rb
   44 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/tag.rb
   45 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/log.rb
   46 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph/vertex.rb
   47 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/dependency_graph.rb
   48 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/state.rb
   49 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/modules/specification_provider.rb
   50 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/delegates/resolution_state.rb
   51 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/delegates/specification_provider.rb
   52 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/resolution.rb
   53 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/resolver.rb
   54 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo/modules/ui.rb
   55 /usr/share/rubygems/rubygems/resolver/molinillo/lib/molinillo.rb
   56 /usr/share/rubygems/rubygems/resolver/molinillo.rb
   57 /usr/share/rubygems/rubygems/resolver/activation_request.rb
   58 /usr/share/rubygems/rubygems/resolver/conflict.rb
   59 /usr/share/rubygems/rubygems/resolver/dependency_request.rb
   60 /usr/share/rubygems/rubygems/resolver/requirement_list.rb
   61 /usr/share/rubygems/rubygems/resolver/stats.rb
   62 /usr/share/rubygems/rubygems/resolver/set.rb
   63 /usr/share/rubygems/rubygems/resolver/api_set.rb
   64 /usr/share/rubygems/rubygems/resolver/composed_set.rb
   65 /usr/share/rubygems/rubygems/resolver/best_set.rb
   66 /usr/share/rubygems/rubygems/resolver/current_set.rb
   67 /usr/share/rubygems/rubygems/resolver/git_set.rb
   68 /usr/share/rubygems/rubygems/resolver/index_set.rb
   69 /usr/share/rubygems/rubygems/resolver/installer_set.rb
   70 /usr/share/rubygems/rubygems/resolver/lock_set.rb
   71 /usr/share/rubygems/rubygems/resolver/vendor_set.rb
   72 /usr/share/rubygems/rubygems/resolver/source_set.rb
   73 /usr/share/rubygems/rubygems/resolver/specification.rb
   74 /usr/share/rubygems/rubygems/resolver/spec_specification.rb
   75 /usr/share/rubygems/rubygems/resolver/api_specification.rb
   76 /usr/share/rubygems/rubygems/resolver/git_specification.rb
   77 /usr/share/rubygems/rubygems/resolver/index_specification.rb
   78 /usr/share/rubygems/rubygems/resolver/installed_specification.rb
   79 /usr/share/rubygems/rubygems/resolver/local_specification.rb
   80 /usr/share/rubygems/rubygems/resolver/lock_specification.rb
   81 /usr/share/rubygems/rubygems/resolver/vendor_specification.rb
   82 /usr/share/rubygems/rubygems/resolver.rb
   83 /usr/share/ruby/uri/rfc2396_parser.rb
   84 /usr/share/ruby/uri/rfc3986_parser.rb
   85 /usr/share/ruby/uri/common.rb
   86 /usr/share/ruby/uri/generic.rb
   87 /usr/share/ruby/uri/ftp.rb
   88 /usr/share/ruby/uri/http.rb
   89 /usr/share/ruby/uri/https.rb
   90 /usr/share/ruby/uri/ldap.rb
   91 /usr/share/ruby/uri/ldaps.rb
   92 /usr/share/ruby/uri/mailto.rb
   93 /usr/share/ruby/uri.rb
   94 /usr/share/rubygems/rubygems/source/git.rb
   95 /usr/share/rubygems/rubygems/source/installed.rb
   96 /usr/share/rubygems/rubygems/source/specific_file.rb
   97 /usr/share/rubygems/rubygems/source/local.rb
   98 /usr/share/rubygems/rubygems/source/lock.rb
   99 /usr/share/rubygems/rubygems/source/vendor.rb
  100 /usr/share/rubygems/rubygems/source.rb
  101 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/version.rb
  102 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/comparable_version.rb
  103 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/ruby_features.rb
  104 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support.rb
  105 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/caller_filter.rb
  106 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/version.rb
  107 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/warnings.rb
  108 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/warnings.rb
  109 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/set.rb
  110 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/flat_map.rb
  111 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/filter_manager.rb
  112 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/dsl.rb
  113 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/console_codes.rb
  114 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/snippet_extractor.rb
  115 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/syntax_highlighter.rb
  116 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/encoded_string.rb
  117 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/exception_presenter.rb
  118 /usr/share/ruby/shellwords.rb
  119 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/shell_escape.rb
  120 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/helpers.rb
  121 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/notifications.rb
  122 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb
  123 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb
  124 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/reentrant_mutex.rb
  125 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/memoized_helpers.rb
  126 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/metadata.rb
  127 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/metadata_filter.rb
  128 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/pending.rb
  129 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/directory_maker.rb
  130 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters.rb
  131 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/ordering.rb
  132 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/world.rb
  133 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/backtrace_formatter.rb
  134 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/ruby_project.rb
  135 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/deprecation_formatter.rb
  136 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/output_wrapper.rb
  137 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/configuration.rb
  138 /usr/share/ruby/optparse.rb
  139 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/option_parser.rb
  140 /usr/lib64/ruby/cgi/escape.so
  141 /usr/share/ruby/cgi/util.rb
  142 /usr/lib64/ruby/strscan.so
  143 /usr/share/ruby/erb.rb
  144 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/configuration_options.rb
  145 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb
  146 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/invocations.rb
  147 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/example.rb
  148 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/shared_example_group.rb
  149 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/recursive_const_methods.rb
  150 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb
  151 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core.rb
  152 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/version.rb
  153 /usr/share/ruby/delegate.rb
  154 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/synchronized_delegator.rb
  155 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe.rb
  156 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/support/threads.rb
  157 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/support/threadsafe_test.rb
  158 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/spec/spec_helper.rb
  159 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/base_formatter.rb
  160 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/base_text_formatter.rb
  161 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/formatters/progress_formatter.rb
  162 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/instance_method_stasher.rb
  163 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/method_double.rb
  164 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/matcher_definition.rb
  165 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/argument_matchers.rb
  166 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/object_reference.rb
  167 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/example_methods.rb
  168 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/proxy.rb
  169 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/test_double.rb
  170 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/fuzzy_matcher.rb
  171 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/argument_list_matcher.rb
  172 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/message_expectation.rb
  173 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/order_group.rb
  174 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/object_formatter.rb
  175 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/error_generator.rb
  176 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/space.rb
  177 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/mutate_const.rb
  178 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/targets.rb
  179 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/syntax.rb
  180 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/configuration.rb
  181 /usr/share/gems/gems/rspec-support-3.7.0/lib/rspec/support/method_signature_verifier.rb
  182 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/verifying_message_expectation.rb
  183 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/method_reference.rb
  184 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/verifying_proxy.rb
  185 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/verifying_double.rb
  186 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks/version.rb
  187 /usr/share/gems/gems/rspec-mocks-3.7.0/lib/rspec/mocks.rb
  188 /usr/share/gems/gems/rspec-core-3.7.1/lib/rspec/core/mocking_adapters/rspec.rb
  189 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/english_phrasing.rb
  190 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/composable.rb
  191 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/built_in/base_matcher.rb
  192 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/built_in.rb
  193 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/generated_descriptions.rb
  194 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/dsl.rb
  195 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/matcher_delegator.rb
  196 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/aliased_matcher.rb
  197 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/expecteds_for_multiple_diffs.rb
  198 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers.rb
  199 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/expectation_target.rb
  200 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/syntax.rb
  201 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/configuration.rb
  202 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/fail_with.rb
  203 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/handler.rb
  204 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations/version.rb
  205 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/expectations.rb
  206 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/non_concurrent_cache_backend.rb
  207 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/mri_cache_backend.rb
  208 /builddir/build/BUILD/thread_safe-0.3.6/usr/share/gems/gems/thread_safe-0.3.6/lib/thread_safe/cache.rb
  209 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/built_in/eq.rb
  210 /usr/share/gems/gems/rspec-expectations-3.7.0/lib/rspec/matchers/built_in/be.rb
* Process memory map:
5577adbc7000-5577adbc8000 r-xp 00000000 09:01 20581122                   /usr/bin/ruby-mri
5577addc7000-5577addc8000 r--p 00000000 09:01 20581122                   /usr/bin/ruby-mri
5577addc8000-5577addc9000 rw-p 00001000 09:01 20581122                   /usr/bin/ruby-mri
5577ae9e0000-5577b08f9000 rw-p 00000000 00:00 0                          [heap]
7f7bb0000000-7f7bb0098000 rw-p 00000000 00:00 0 
7f7bb0098000-7f7bb4000000 ---p 00000000 00:00 0 
7f7bb8000000-7f7bb8067000 rw-p 00000000 00:00 0 
7f7bb8067000-7f7bbc000000 ---p 00000000 00:00 0 
7f7bbc000000-7f7bbc067000 rw-p 00000000 00:00 0 
7f7bbc067000-7f7bc0000000 ---p 00000000 00:00 0 
7f7bc0000000-7f7bc00af000 rw-p 00000000 00:00 0 
7f7bc00af000-7f7bc4000000 ---p 00000000 00:00 0 
7f7bc4000000-7f7bc40a2000 rw-p 00000000 00:00 0 
7f7bc40a2000-7f7bc8000000 ---p 00000000 00:00 0 
7f7bc8000000-7f7bc8067000 rw-p 00000000 00:00 0 
7f7bc8067000-7f7bcc000000 ---p 00000000 00:00 0 
7f7bcc000000-7f7bcc067000 rw-p 00000000 00:00 0 
7f7bcc067000-7f7bd0000000 ---p 00000000 00:00 0 
7f7bd0000000-7f7bd0067000 rw-p 00000000 00:00 0 
7f7bd0067000-7f7bd4000000 ---p 00000000 00:00 0 
7f7bd4000000-7f7bd4117000 rw-p 00000000 00:00 0 
7f7bd4117000-7f7bd8000000 ---p 00000000 00:00 0 
7f7bd8000000-7f7bd8067000 rw-p 00000000 00:00 0 
7f7bd8067000-7f7bdc000000 ---p 00000000 00:00 0 
7f7be0000000-7f7be0060000 rw-p 00000000 00:00 0 
7f7be0060000-7f7be4000000 ---p 00000000 00:00 0 
7f7be5c7d000-7f7be5e80000 r--s 00000000 09:01 20713178                   /usr/lib64/libc-2.26.9000.so
7f7be5e80000-7f7be613c000 r--s 00000000 09:01 20714054                   /usr/lib64/libruby.so.2.5.0
7f7be613c000-7f7be653e000 rw-p 00000000 00:00 0 
7f7be6569000-7f7be657f000 r-xp 00000000 09:01 20713166                   /usr/lib64/libgcc_s-7-20180104.so.1
7f7be657f000-7f7be677e000 ---p 00016000 09:01 20713166                   /usr/lib64/libgcc_s-7-20180104.so.1
7f7be677e000-7f7be677f000 r--p 00015000 09:01 20713166                   /usr/lib64/libgcc_s-7-20180104.so.1
7f7be677f000-7f7be6780000 rw-p 00016000 09:01 20713166                   /usr/lib64/libgcc_s-7-20180104.so.1
7f7be6780000-7f7be8000000 rw-p 00000000 00:00 0 
7f7be8000000-7f7be8067000 rw-p 00000000 00:00 0 
7f7be8067000-7f7bec000000 ---p 00000000 00:00 0 
7f7bec000000-7f7bec069000 rw-p 00000000 00:00 0 
7f7bec069000-7f7bf0000000 ---p 00000000 00:00 0 
7f7bf0000000-7f7bf0067000 rw-p 00000000 00:00 0 
7f7bf0067000-7f7bf4000000 ---p 00000000 00:00 0 
7f7bf4000000-7f7bf4067000 rw-p 00000000 00:00 0 
7f7bf4067000-7f7bf8000000 ---p 00000000 00:00 0 
7f7bf8000000-7f7bf80af000 rw-p 00000000 00:00 0 
7f7bf80af000-7f7bfc000000 ---p 00000000 00:00 0 
7f7bfc000000-7f7bfc067000 rw-p 00000000 00:00 0 
7f7bfc067000-7f7c00000000 ---p 00000000 00:00 0 
7f7c00000000-7f7c00067000 rw-p 00000000 00:00 0 
7f7c00067000-7f7c04000000 ---p 00000000 00:00 0 
7f7c04000000-7f7c040a3000 rw-p 00000000 00:00 0 
7f7c040a3000-7f7c08000000 ---p 00000000 00:00 0 
7f7c08000000-7f7c08067000 rw-p 00000000 00:00 0 
7f7c08067000-7f7c0c000000 ---p 00000000 00:00 0 
7f7c0c000000-7f7c0c067000 rw-p 00000000 00:00 0 
7f7c0c067000-7f7c10000000 ---p 00000000 00:00 0 
7f7c10000000-7f7c10093000 rw-p 00000000 00:00 0 
7f7c10093000-7f7c14000000 ---p 00000000 00:00 0 
7f7c14000000-7f7c14073000 rw-p 00000000 00:00 0 
7f7c14073000-7f7c18000000 ---p 00000000 00:00 0 
7f7c18000000-7f7c18067000 rw-p 00000000 00:00 0 
7f7c18067000-7f7c1c000000 ---p 00000000 00:00 0 
7f7c1c000000-7f7c1c0f9000 rw-p 00000000 00:00 0 
7f7c1c0f9000-7f7c20000000 ---p 00000000 00:00 0 
7f7c20000000-7f7c200a0000 rw-p 00000000 00:00 0 
7f7c200a0000-7f7c24000000 ---p 00000000 00:00 0 
7f7c24000000-7f7c24073000 rw-p 00000000 00:00 0 
7f7c24073000-7f7c28000000 ---p 00000000 00:00 0 
7f7c28000000-7f7c28073000 rw-p 00000000 00:00 0 
7f7c28073000-7f7c2c000000 ---p 00000000 00:00 0 
7f7c2c000000-7f7c2c060000 rw-p 00000000 00:00 0 
7f7c2c060000-7f7c30000000 ---p 00000000 00:00 0 
7f7c30000000-7f7c30069000 rw-p 00000000 00:00 0 
7f7c30069000-7f7c34000000 ---p 00000000 00:00 0 
7f7c34000000-7f7c34067000 rw-p 00000000 00:00 0 
7f7c34067000-7f7c38000000 ---p 00000000 00:00 0 
7f7c38000000-7f7c38067000 rw-p 00000000 00:00 0 
7f7c38067000-7f7c3c000000 ---p 00000000 00:00 0 
7f7c3c000000-7f7c3c069000 rw-p 00000000 00:00 0 
7f7c3c069000-7f7c40000000 ---p 00000000 00:00 0 
7f7c40000000-7f7c400a9000 rw-p 00000000 00:00 0 
7f7c400a9000-7f7c44000000 ---p 00000000 00:00 0 
7f7c44000000-7f7c44069000 rw-p 00000000 00:00 0 
7f7c44069000-7f7c48000000 ---p 00000000 00:00 0 
7f7c48000000-7f7c48067000 rw-p 00000000 00:00 0 
7f7c48067000-7f7c4c000000 ---p 00000000 00:00 0 
7f7c4c000000-7f7c4c067000 rw-p 00000000 00:00 0 
7f7c4c067000-7f7c50000000 ---p 00000000 00:00 0 
7f7c50000000-7f7c50067000 rw-p 00000000 00:00 0 
7f7c50067000-7f7c54000000 ---p 00000000 00:00 0 
7f7c54000000-7f7c54067000 rw-p 00000000 00:00 0 
7f7c54067000-7f7c58000000 ---p 00000000 00:00 0 
7f7c580ae000-7f7c586ce000 rw-p 00000000 00:00 0 
7f7c586ce000-7f7c587c8000 rw-p 00000000 00:00 0 
7f7c587c8000-7f7c587c9000 ---p 00000000 00:00 0 
7f7c587c9000-7f7c589ca000 rw-p 00000000 00:00 0 
7f7c589ca000-7f7c589cb000 ---p 00000000 00:00 0 
7f7c589cb000-7f7c58bcc000 rw-p 00000000 00:00 0 
7f7c58bcc000-7f7c58bcd000 ---p 00000000 00:00 0 
7f7c58bcd000-7f7c58dce000 rw-p 00000000 00:00 0 
7f7c58dce000-7f7c58dcf000 ---p 00000000 00:00 0 
7f7c58dcf000-7f7c58fd0000 rw-p 00000000 00:00 0 
7f7c58fd0000-7f7c58fd1000 ---p 00000000 00:00 0 
7f7c58fd1000-7f7c591d2000 rw-p 00000000 00:00 0 
7f7c591d2000-7f7c591d3000 ---p 00000000 00:00 0 
7f7c591d3000-7f7c593d4000 rw-p 00000000 00:00 0 
7f7c593d4000-7f7c593d5000 ---p 00000000 00:00 0 
7f7c593d5000-7f7c595d6000 rw-p 00000000 00:00 0 
7f7c595d6000-7f7c595d7000 ---p 00000000 00:00 0 
7f7c595d7000-7f7c597d8000 rw-p 00000000 00:00 0 
7f7c597d8000-7f7c597d9000 ---p 00000000 00:00 0 
7f7c597d9000-7f7c599da000 rw-p 00000000 00:00 0 
7f7c599da000-7f7c599db000 ---p 00000000 00:00 0 
7f7c599db000-7f7c59bdc000 rw-p 00000000 00:00 0 
7f7c59bdc000-7f7c59bdd000 ---p 00000000 00:00 0 
7f7c59bdd000-7f7c59dde000 rw-p 00000000 00:00 0 
7f7c59dde000-7f7c59ddf000 ---p 00000000 00:00 0 
7f7c59ddf000-7f7c59fe0000 rw-p 00000000 00:00 0 
7f7c59fe0000-7f7c59fe1000 ---p 00000000 00:00 0 
7f7c59fe1000-7f7c5a1e2000 rw-p 00000000 00:00 0 
7f7c5a1e2000-7f7c5a1e3000 ---p 00000000 00:00 0 
7f7c5a1e3000-7f7c5a3e4000 rw-p 00000000 00:00 0 
7f7c5a3e4000-7f7c5a3e5000 ---p 00000000 00:00 0 
7f7c5a3e5000-7f7c5a5e6000 rw-p 00000000 00:00 0 
7f7c5a5e6000-7f7c5a5e7000 ---p 00000000 00:00 0 
7f7c5a5e7000-7f7c5a7e8000 rw-p 00000000 00:00 0 
7f7c5a7e8000-7f7c5a7e9000 ---p 00000000 00:00 0 
7f7c5a7e9000-7f7c5a9ea000 rw-p 00000000 00:00 0 
7f7c5a9ea000-7f7c5a9eb000 ---p 00000000 00:00 0 
7f7c5a9eb000-7f7c5abec000 rw-p 00000000 00:00 0 
7f7c5abec000-7f7c5abed000 ---p 00000000 00:00 0 
7f7c5abed000-7f7c5adee000 rw-p 00000000 00:00 0 
7f7c5adee000-7f7c5adef000 ---p 00000000 00:00 0 
7f7c5adef000-7f7c5aff0000 rw-p 00000000 00:00 0 
7f7c5aff0000-7f7c5aff1000 ---p 00000000 00:00 0 
7f7c5aff1000-7f7c5b1f2000 rw-p 00000000 00:00 0 
7f7c5b1f2000-7f7c5b1f3000 ---p 00000000 00:00 0 
7f7c5b1f3000-7f7c5b3f4000 rw-p 00000000 00:00 0 
7f7c5b3f4000-7f7c5b3f5000 ---p 00000000 00:00 0 
7f7c5b3f5000-7f7c5b5f6000 rw-p 00000000 00:00 0 
7f7c5b5f6000-7f7c5b5f7000 ---p 00000000 00:00 0 
7f7c5b5f7000-7f7c5b7f8000 rw-p 00000000 00:00 0 
7f7c5b7f8000-7f7c5b7f9000 ---p 00000000 00:00 0 
7f7c5b7f9000-7f7c5b9fa000 rw-p 00000000 00:00 0 
7f7c5b9fa000-7f7c5b9fb000 ---p 00000000 00:00 0 
7f7c5b9fb000-7f7c5bbfc000 rw-p 00000000 00:00 0 
7f7c5bbfc000-7f7c5bbfd000 ---p 00000000 00:00 0 
7f7c5bbfd000-7f7c5bdfe000 rw-p 00000000 00:00 0 
7f7c5bdfe000-7f7c5bdff000 ---p 00000000 00:00 0 
7f7c5bdff000-7f7c5beff000 rw-p 00000000 00:00 0 
7f7c5beff000-7f7c5bf00000 ---p 00000000 00:00 0 
7f7c5bf00000-7f7c5c000000 rw-p 00000000 00:00 0 
7f7c5c000000-7f7c5c05d000 rw-p 00000000 00:00 0 
7f7c5c05d000-7f7c60000000 ---p 00000000 00:00 0 
7f7c6006b000-7f7c6016c000 rw-p 00000000 00:00 0 
7f7c6016c000-7f7c6016d000 ---p 00000000 00:00 0 
7f7c6016d000-7f7c6036e000 rw-p 00000000 00:00 0 
7f7c6036e000-7f7c6036f000 ---p 00000000 00:00 0 
7f7c6036f000-7f7c60570000 rw-p 00000000 00:00 0 
7f7c60570000-7f7c60571000 ---p 00000000 00:00 0 
7f7c60571000-7f7c60772000 rw-p 00000000 00:00 0 
7f7c60772000-7f7c60773000 ---p 00000000 00:00 0 
7f7c60773000-7f7c60974000 rw-p 00000000 00:00 0 
7f7c60974000-7f7c60975000 ---p 00000000 00:00 0 
7f7c60975000-7f7c60b76000 rw-p 00000000 00:00 0 
7f7c60b76000-7f7c60b77000 ---p 00000000 00:00 0 
7f7c60b77000-7f7c60d78000 rw-p 00000000 00:00 0 
7f7c60d78000-7f7c60d79000 ---p 00000000 00:00 0 
7f7c60d79000-7f7c60f7a000 rw-p 00000000 00:00 0 
7f7c60f7a000-7f7c60f7b000 ---p 00000000 00:00 0 
7f7c60f7b000-7f7c6117c000 rw-p 00000000 00:00 0 
7f7c6117c000-7f7c6117d000 ---p 00000000 00:00 0 
7f7c6117d000-7f7c6147f000 rw-p 00000000 00:00 0 
7f7c6147f000-7f7c61480000 ---p 00000000 00:00 0 
7f7c61480000-7f7c61681000 rw-p 00000000 00:00 0 
7f7c61681000-7f7c61682000 ---p 00000000 00:00 0 
7f7c61682000-7f7c61883000 rw-p 00000000 00:00 0 
7f7c61883000-7f7c61888000 r-xp 00000000 09:01 20714078                   /usr/lib64/ruby/strscan.so
7f7c61888000-7f7c61a87000 ---p 00005000 09:01 20714078                   /usr/lib64/ruby/strscan.so
7f7c61a87000-7f7c61a88000 r--p 00004000 09:01 20714078                   /usr/lib64/ruby/strscan.so
7f7c61a88000-7f7c61a89000 rw-p 00000000 00:00 0 
7f7c61a89000-7f7c61a8c000 r-xp 00000000 09:01 20714057                   /usr/lib64/ruby/cgi/escape.so
7f7c61a8c000-7f7c61c8b000 ---p 00003000 09:01 20714057                   /usr/lib64/ruby/cgi/escape.so
7f7c61c8b000-7f7c61c8c000 r--p 00002000 09:01 20714057                   /usr/lib64/ruby/cgi/escape.so
7f7c61c8c000-7f7c61c8d000 rw-p 00000000 00:00 0 
7f7c61c8d000-7f7c61c94000 r-xp 00000000 09:01 20714077                   /usr/lib64/ruby/stringio.so
7f7c61c94000-7f7c61e94000 ---p 00007000 09:01 20714077                   /usr/lib64/ruby/stringio.so
7f7c61e94000-7f7c61e95000 r--p 00007000 09:01 20714077                   /usr/lib64/ruby/stringio.so
7f7c61e95000-7f7c61e96000 rw-p 00000000 00:00 0 
7f7c61e96000-7f7c61e98000 r-xp 00000000 09:01 20973246                   /usr/lib64/ruby/enc/trans/transdb.so
7f7c61e98000-7f7c62098000 ---p 00002000 09:01 20973246                   /usr/lib64/ruby/enc/trans/transdb.so
7f7c62098000-7f7c62099000 r--p 00002000 09:01 20973246                   /usr/lib64/ruby/enc/trans/transdb.so
7f7c62099000-7f7c6209a000 rw-p 00000000 00:00 0 
7f7c6209a000-7f7c6209c000 r-xp 00000000 09:01 20847551                   /usr/lib64/ruby/enc/encdb.so
7f7c6209c000-7f7c6229b000 ---p 00002000 09:01 20847551                   /usr/lib64/ruby/enc/encdb.so
7f7c6229b000-7f7c6229c000 r--p 00001000 09:01 20847551                   /usr/lib64/ruby/enc/encdb.so
7f7c6229c000-7f7c6229d000 rw-p 00000000 00:00 0 
7f7c6229d000-7f7c6244f000 r-xp 00000000 09:01 20713178                   /usr/lib64/libc-2.26.9000.so
7f7c6244f000-7f7c6264f000 ---p 001b2000 09:01 20713178                   /usr/lib64/libc-2.26.9000.so
7f7c6264f000-7f7c62653000 r--p 001b2000 09:01 20713178                   /usr/lib64/libc-2.26.9000.so
7f7c62653000-7f7c62655000 rw-p 001b6000 09:01 20713178                   /usr/lib64/libc-2.26.9000.so
7f7c62655000-7f7c62659000 rw-p 00000000 00:00 0 
7f7c62659000-7f7c627eb000 r-xp 00000000 09:01 20713184                   /usr/lib64/libm-2.26.9000.so
7f7c627eb000-7f7c629ea000 ---p 00192000 09:01 20713184                   /usr/lib64/libm-2.26.9000.so
7f7c629ea000-7f7c629eb000 r--p 00191000 09:01 20713184                   /usr/lib64/libm-2.26.9000.so
7f7c629eb000-7f7c629ec000 rw-p 00192000 09:01 20713184                   /usr/lib64/libm-2.26.9000.so
7f7c629ec000-7f7c629f5000 r-xp 00000000 09:01 20713503                   /usr/lib64/libcrypt-2.26.9000.so
7f7c629f5000-7f7c62bf4000 ---p 00009000 09:01 20713503                   /usr/lib64/libcrypt-2.26.9000.so
7f7c62bf4000-7f7c62bf5000 r--p 00008000 09:01 20713503                   /usr/lib64/libcrypt-2.26.9000.so
7f7c62bf5000-7f7c62bf6000 rw-p 00009000 09:01 20713503                   /usr/lib64/libcrypt-2.26.9000.so
7f7c62bf6000-7f7c62c24000 rw-p 00000000 00:00 0 
7f7c62c24000-7f7c62c27000 r-xp 00000000 09:01 20713182                   /usr/lib64/libdl-2.26.9000.so
7f7c62c27000-7f7c62e26000 ---p 00003000 09:01 20713182                   /usr/lib64/libdl-2.26.9000.so
7f7c62e26000-7f7c62e27000 r--p 00002000 09:01 20713182                   /usr/lib64/libdl-2.26.9000.so
7f7c62e27000-7f7c62e28000 rw-p 00003000 09:01 20713182                   /usr/lib64/libdl-2.26.9000.so
7f7c62e28000-7f7c62e41000 r-xp 00000000 09:01 20713194                   /usr/lib64/libpthread-2.26.9000.so
7f7c62e41000-7f7c63041000 ---p 00019000 09:01 20713194                   /usr/lib64/libpthread-2.26.9000.so
7f7c63041000-7f7c63042000 r--p 00019000 09:01 20713194                   /usr/lib64/libpthread-2.26.9000.so
7f7c63042000-7f7c63043000 rw-p 0001a000 09:01 20713194                   /usr/lib64/libpthread-2.26.9000.so
7f7c63043000-7f7c63047000 rw-p 00000000 00:00 0 
7f7c63047000-7f7c632e9000 r-xp 00000000 09:01 20714054                   /usr/lib64/libruby.so.2.5.0
7f7c632e9000-7f7c634e8000 ---p 002a2000 09:01 20714054                   /usr/lib64/libruby.so.2.5.0
7f7c634e8000-7f7c634f0000 r--p 002a1000 09:01 20714054                   /usr/lib64/libruby.so.2.5.0
7f7c634f0000-7f7c634f1000 rw-p 002a9000 09:01 20714054                   /usr/lib64/libruby.so.2.5.0
7f7c634f1000-7f7c63501000 rw-p 00000000 00:00 0 
7f7c63501000-7f7c63526000 r-xp 00000000 09:01 20713171                   /usr/lib64/ld-2.26.9000.so
7f7c6358f000-7f7c635b5000 r--s 00000000 09:01 20713194                   /usr/lib64/libpthread-2.26.9000.so
7f7c635b5000-7f7c63616000 rw-p 00000000 00:00 0 
7f7c63616000-7f7c63617000 ---p 00000000 00:00 0 
7f7c63617000-7f7c63722000 rw-p 00000000 00:00 0 
7f7c63724000-7f7c63726000 r--s 00000000 09:01 20581122                   /usr/bin/ruby-mri
7f7c63726000-7f7c63727000 r--p 00025000 09:01 20713171                   /usr/lib64/ld-2.26.9000.so
7f7c63727000-7f7c63728000 rw-p 00026000 09:01 20713171                   /usr/lib64/ld-2.26.9000.so
7f7c63728000-7f7c63729000 rw-p 00000000 00:00 0 
7fff75815000-7fff76014000 rw-p 00000000 00:00 0                          [stack]
7fff76042000-7fff76045000 r--p 00000000 00:00 0                          [vvar]
7fff76045000-7fff76047000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

[normalperson (Eric Wong)]

[e-mail] wrote:

https://bugs.ruby-lang.org/issues/14357

The thread_safe gem is not maintained anymore, but I don't see
any reason why its test suite should segfault with Ruby 2.5.

Right, no 3rd-party C exts loaded and I hit this in trunk, too.
Using -fsanitize=address reveals use-after-free in st.c
Investigating, but maybe Vladimir can find it sooner.

thread_safe-0.3.6/spec/spec_helper.rb:5:in `<top (required)>': [DEPRECATION] ::[] is deprecated. Use ::new instead.

Randomized with seed 18515
......................................................=================================================================
==18224==ERROR: AddressSanitizer: heap-use-after-free on address 0x6230002112c0 at pc 0x557ae852ae34 bp 0x7fb3c088f5c0 sp 0x7fb3c088f5b8
READ of size 8 at 0x6230002112c0 thread T332 (cache_loops_sp*)
#0 0x557ae852ae33 in find_table_entry_ind ../st.c:873
#1 0x557ae852f847 in st_lookup ../st.c:1049
#2 0x557ae831139e in rb_hash_aref ../hash.c:853
#3 0x557ae8648e27 in vm_opt_aref ../vm_insnhelper.c:3650
#4 0x557ae8648e27 in vm_exec_core $SRC/ruby/insns.def:1175
#5 0x557ae8651696 in vm_exec ../vm.c:1791
#6 0x557ae8654272 in invoke_block ../vm.c:994
#7 0x557ae8654272 in invoke_iseq_block_from_c ../vm.c:1046
#8 0x557ae8669c22 in invoke_block_from_c_bh ../vm.c:1064
#9 0x557ae8669c22 in vm_yield ../vm.c:1109
#10 0x557ae8669c22 in rb_yield_0 ../vm_eval.c:970
#11 0x557ae8669c22 in rb_yield_1 ../vm_eval.c:976
#12 0x557ae83a0a95 in int_dotimes ../numeric.c:4984
#13 0x557ae862da57 in vm_call_cfunc_with_frame ../vm_insnhelper.c:1921
#14 0x557ae862da57 in vm_call_cfunc ../vm_insnhelper.c:1937
#15 0x557ae8646213 in vm_exec_core $SRC/ruby/insns.def:719
#16 0x557ae8651696 in vm_exec ../vm.c:1791
#17 0x557ae8654272 in invoke_block ../vm.c:994
#18 0x557ae8654272 in invoke_iseq_block_from_c ../vm.c:1046
#19 0x557ae8658126 in invoke_block_from_c_proc ../vm.c:1139
#20 0x557ae8658126 in vm_invoke_proc ../vm.c:1157
#21 0x557ae8658126 in rb_vm_invoke_proc ../vm.c:1178
#22 0x557ae85a95e3 in thread_do_start ../thread.c:603
#23 0x557ae85a95e3 in thread_start_func_2 ../thread.c:647
#24 0x557ae85aa680 in thread_start_func_1 ../thread_pthread.c:872
#25 0x7fb3d2fb6063 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8063)
#26 0x7fb3d231662c in clone (/lib/x86_64-linux-gnu/libc.so.6+0xe862c)

0x6230002112c0 is located 2496 bytes inside of 6144-byte region [0x623000210900,0x623000212100)
freed by thread T343 (cache_loops_sp*) here:
#0 0x7fb3d3222527 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x54527)
#1 0x557ae8303f06 in objspace_xfree ../gc.c:7987
#2 0x557ae8303f06 in ruby_sized_xfree ../gc.c:8082
#3 0x557ae8303f06 in ruby_xfree ../gc.c:8089

previously allocated by thread T331 (cache_loops_sp*) here:
#0 0x7fb3d322273f in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x5473f)
#1 0x557ae82fd573 in objspace_xmalloc0 ../gc.c:7927

Thread T332 (cache_loops_sp*) created by T0 here:
#0 0x7fb3d31f1bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
#1 0x557ae859489d in native_thread_create ../thread_pthread.c:1008
#2 0x557ae859489d in thread_create_core ../thread.c:757
#3 0x557ae884894c ($SRC/ruby/a/i/bin/ruby+0x63f94c)

Thread T343 (cache_loops_sp*) created by T0 here:
#0 0x7fb3d31f1bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
#1 0x557ae859489d in native_thread_create ../thread_pthread.c:1008
#2 0x557ae859489d in thread_create_core ../thread.c:757
#3 0x557ae884894c ($SRC/ruby/a/i/bin/ruby+0x63f94c)

Thread T331 (cache_loops_sp*) created by T0 here:
#0 0x7fb3d31f1bba in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x23bba)
#1 0x557ae859489d in native_thread_create ../thread_pthread.c:1008
#2 0x557ae859489d in thread_create_core ../thread.c:757
#3 0x557ae884894c ($SRC/ruby/a/i/bin/ruby+0x63f94c)

SUMMARY: AddressSanitizer: heap-use-after-free ../st.c:873 find_table_entry_ind
Shadow bytes around the buggy address:
0x0c468003a200: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a210: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a220: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a230: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a240: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c468003a250: fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd
0x0c468003a260: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a270: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a280: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a290: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c468003a2a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Contiguous container OOB:fc
ASan internal: fe
==18224==ABORTING

[normalperson (Eric Wong)]

Eric Wong wrote:

Using -fsanitize=address reveals use-after-free in st.c
Investigating, but maybe Vladimir can find it sooner.

ST_DEBUG doesn't reveal anything nor does switching to
QUADRATIC_PROBE. Waiting for valgrind which is very slow
and I need to go afk to do other things...

[normalperson (Eric Wong)]

Eric Wong wrote:

ST_DEBUG doesn't reveal anything nor does switching to
QUADRATIC_PROBE. Waiting for valgrind which is very slow
and I need to go afk to do other things...

Could be a GC problem related to coverage (also [Bug #14334])
since there's minimal changes in st.c from v2_4_0..v2_5_0 and
I'm not hitting it with the 2.4 branch.

Nothing jumped out at me from reading diffs, I don't think
the guard I proposed in [ruby-core:85352] is effective, even

Bisecting now, and going mostly away from computers for a bit...

git bisect start v2_5_0 v2_4_0
git bisect run /tmp/bisect.sh

==> /tmp/bisect.sh <==
#!/bin/sh
git clean -qfdx || exit 125
autoconf >/dev/null || exit 125
pfx=/tmp/bisect
rm -rf $pfx || exit 125
./configure
--prefix=$pfx --disable-install-doc >/dev/null || exit 125
make -j8 >/dev/null || exit 125
make update-gems || :
make -j8 install >/dev/null || exit 125
export PATH="$pfx/bin:$PATH"
(
# copied a bunch of gems over to /tmp after
# gem install rspec rspec-expectations coveralls
cd /tmp/gems && gem install *.gem
)
gem unpack /tmp/gems/thread_safe-0.3.6.gem || exit 125
cd thread_safe-0.3.6 || exit 125

run twice, sometimes won't crash

rspec -I lib spec || exit 1
rspec -I lib spec || exit 1

EW

[normalperson (Eric Wong)]

Eric Wong [e-mail] wrote:

Could be a GC problem related to coverage (also [Bug #14334])
since there's minimal changes in st.c from v2_4_0..v2_5_0 and
I'm not hitting it with the 2.4 branch.

Or not... Hard to tell if I'm chasing multiple bugs, here.

[normalperson (Eric Wong)]

Eric Wong [e-mail] wrote:

Could be a GC problem related to coverage (also [Bug #14334])

Nope, not a fault of coverage. Commented simplecov and
coveralls from spec/spec_helper.rb in thread_safe-0.3.6 and
asan still chokes...

[normalperson (Eric Wong)]

Eric Wong [e-mail] wrote:

[e-mail] wrote:

https://bugs.ruby-lang.org/issues/14357

The thread_safe gem is not maintained anymore, but I don't see
any reason why its test suite should segfault with Ruby 2.5.

Right, no 3rd-party C exts loaded and I hit this in trunk, too.
Using -fsanitize=address reveals use-after-free in st.c
Investigating, but maybe Vladimir can find it sooner.

Maybe my initial investigation was correct, after all.

valgrind takes forever, but indicates the free is caused by
rebuild_table; so it doesn't look like we missed GC marking
during rebuild. Disabling the free(tab->entries) at line
st.c:792 (patch below) seems to indicate success with the
thread_safe test suite (letting it loop overnight).

Looks like the new_tab != tab case of rebuild is leaving a
hanging reference somewhere.

==9885== Thread 32 cache_loops_sp*:
==9885== Invalid read of size 8
==9885== at 0x235622: find_table_entry_ind (st.c:873)
==9885== by 0x236C95: st_lookup (st.c:1049)
==9885== by 0x1520CE: rb_hash_aref (hash.c:853)
==9885== by 0x2A95E0: vm_opt_aref (vm_insnhelper.c:3650)
==9885== by 0x2A95E0: vm_exec_core (insns.def:1175)
==9885== by 0x2ACA83: vm_exec (vm.c:1790)
==9885== by 0x2AD875: invoke_block (vm.c:993)
==9885== by 0x2AD875: invoke_iseq_block_from_c (vm.c:1045)
==9885== by 0x2B64A8: invoke_block_from_c_bh (vm.c:1063)
==9885== by 0x2B64A8: vm_yield (vm.c:1108)
==9885== by 0x2B64A8: rb_yield_0 (vm_eval.c:970)
==9885== by 0x2B64A8: rb_yield_1 (vm_eval.c:976)
==9885== by 0x19238D: int_dotimes (numeric.c:4984)
==9885== by 0x29F816: vm_call_cfunc_with_frame (vm_insnhelper.c:1921)
==9885== by 0x29F816: vm_call_cfunc (vm_insnhelper.c:1937)
==9885== by 0x2A83D9: vm_exec_core (insns.def:719)
==9885== by 0x2ACA83: vm_exec (vm.c:1790)
==9885== by 0x2AD875: invoke_block (vm.c:993)
==9885== by 0x2AD875: invoke_iseq_block_from_c (vm.c:1045)
==9885== Address 0xbeafe88 is 43,080 bytes inside a block of size 49,152 free'd
==9885== at 0x4C29E90: free (vg_replace_malloc.c:473)
==9885== by 0x14C3EC: objspace_xfree (gc.c:7987)
==9885== by 0x14C3EC: ruby_sized_xfree (gc.c:8082)
==9885== by 0x14C3EC: ruby_xfree (gc.c:8089)
==9885== by 0x236472: rebuild_table (st.c:792)
==9885== by 0x237E85: rebuild_table_if_necessary (st.c:1090)
==9885== by 0x237E85: st_add_direct_with_hash (st.c:1153)
==9885== by 0x237E85: st_update (st.c:1431)
==9885== by 0x150A4E: tbl_update (hash.c:561)
==9885== by 0x150A4E: rb_hash_aset (hash.c:1654)
==9885== by 0x2A9687: vm_opt_aset (vm_insnhelper.c:3671)
==9885== by 0x2A9687: vm_exec_core (insns.def:1189)
==9885== by 0x2ACA83: vm_exec (vm.c:1790)
==9885== by 0x2AD875: invoke_block (vm.c:993)
==9885== by 0x2AD875: invoke_iseq_block_from_c (vm.c:1045)
==9885== by 0x2B674F: invoke_block_from_c_bh (vm.c:1063)
==9885== by 0x2B674F: vm_yield (vm.c:1108)
==9885== by 0x2B674F: rb_yield_0 (vm_eval.c:970)
==9885== by 0x2B674F: rb_yield (vm_eval.c:983)
==9885== by 0x131C86: rb_ensure (eval.c:1035)
==9885== by 0x29F816: vm_call_cfunc_with_frame (vm_insnhelper.c:1921)
==9885== by 0x29F816: vm_call_cfunc (vm_insnhelper.c:1937)
==9885== by 0x2A83D9: vm_exec_core (insns.def:719)

Line numbers based on r62184
(git commit 05c18139a1545a61caaaf33d888c8427d346b571).

Following patch hides the problem by introducing a leak:

--- a/st.c
+++ b/st.c
@@ -789,7 +789,7 @@ rebuild_table(st_table *tab)
	if (tab->bins != NULL)
	    free(tab->bins);
	tab->bins = new_tab->bins;
-	free(tab->entries);
+	/* free(tab->entries); */ /* NOT FOR PRODUCTION USE */
	tab->entries = new_tab->entries;
	free(new_tab);
}

(gdb) up
#17 0x00005604a6dd173d in find_table_entry_ind (tab=tab@entry=0x7f13e4444ac0, hash_value=hash_value@entry=0,
key=key@entry=94578030726560) at ../st.c:874
874 && PTR_EQUAL(tab, &entries[bin - ENTRY_BASE], hash_value, key))
(gdb) up
#18 0x00005604a6dd2d26 in st_lookup (tab=0x7f13e4444ac0, key=key@entry=94578030726560, value=value@entry=0x7f132fdfc2f8)
at ../st.c:1050
1050 bin = find_table_entry_ind(tab, hash, key);
(gdb) p *tab
$1 = {entry_power = 7 '\a', bin_power = 8 '\b', size_ind = 0 '\000', rebuilds_num = 213, type = 0x5604a71ce210 ,
num_entries = 121, bins = 0x7f13e445a340, entries_start = 0, entries_bound = 121, entries = 0x7f13e445c6b0}

Looks like it's freshly rebuilt table. Pretty easy to reproduce
the problem on 2.5, I remember it took more tries on 2.4 (didn't
valgrind). An extra pair of eyes more experienced with this
code than I am would be appreciated. Thanks.

[vmakarov (Vladimir Makarov)]

On 02/06/2018 05:00 AM, Eric Wong wrote:

Eric Wong [e-mail] wrote:

[e-mail] wrote:

https://bugs.ruby-lang.org/issues/14357

The thread_safe gem is not maintained anymore, but I don't see
any reason why its test suite should segfault with Ruby 2.5.
Right, no 3rd-party C exts loaded and I hit this in trunk, too.
Using -fsanitize=address reveals use-after-free in st.c
Investigating, but maybe Vladimir can find it sooner.
Maybe my initial investigation was correct, after all.

valgrind takes forever, but indicates the free is caused by
rebuild_table; so it doesn't look like we missed GC marking
during rebuild. Disabling the free(tab->entries) at line
st.c:792 (patch below) seems to indicate success with the
thread_safe test suite (letting it loop overnight).

Looks like the new_tab != tab case of rebuild is leaving a
hanging reference somewhere.

Looks like it's freshly rebuilt table. Pretty easy to reproduce
the problem on 2.5, I remember it took more tries on 2.4 (didn't
valgrind). An extra pair of eyes more experienced with this
code than I am would be appreciated. Thanks.

Eric, thank you for working on the problem and analyzing it. I'll look
at this and try to fix it as soon as possible.

[normalperson (Eric Wong)]

Vladimir Makarov [e-mail] wrote:

On 02/06/2018 05:00 AM, Eric Wong wrote:

during rebuild. Disabling the free(tab->entries) at line
st.c:792 (patch below) seems to indicate success with the
thread_safe test suite (letting it loop overnight).

It still crashed after four runs :< It might run longer with
the simplecov/coveralls stuff commented out in spec_helper.rb
since coverage creates a giant hash and might increase the chance
of failure.

Eric, thank you for working on the problem and analyzing it. I'll look at
this and try to fix it as soon as possible.

Good to know :)

[vmakarov (Vladimir Makarov)]

On 02/06/2018 02:38 PM, Eric Wong wrote:

Vladimir Makarov [e-mail] wrote:

On 02/06/2018 05:00 AM, Eric Wong wrote:

during rebuild. Disabling the free(tab->entries) at line
st.c:792 (patch below) seems to indicate success with the
thread_safe test suite (letting it loop overnight).
It still crashed after four runs :< It might run longer with
the simplecov/coveralls stuff commented out in spec_helper.rb
since coverage creates a giant hash and might increase the chance
of failure.

Eric, thank you for working on the problem and analyzing it. I'll look at
this and try to fix it as soon as possible.

I reproduced this crash although the reproducing is not stable with or
without valgrind.

It is a typical data race.  The same problem existed in the old hash
tables.  It also rebuilt tables and freed old data structure.

File st.c was never thread-safe.  The data races are/were possible
in many places.

We could make st.c thread-safe.  But I don't think it is a right way. 
It is not a trivial task and it also will hurt performance
considerably.  We still needs thread-unaware level to work with hash
tables (st.c) for cases when tables are used internally in one thread.

So I think the crash should be fixed in other places where calls of st.c
happen.

I don't know how it should be fixed because I don't know Ruby thread
semantics.  Does Ruby guarantee that there are no data races or should a
ruby programmer still provides thread synchronization despite GIL?  If
it is later, thread_safe gem is probably buggy because one thread
reading a table and another thread inserting elements while process
table in a Ruby block.  If there is no sync it is a typical data race
and the result is unpredictable.  In this case it is a segfault crash. 
We could just give a better message about the data races if segfault
happens in st.c.

Also I don't know how GIL works.  Where the thread switching can
happen.  Is the switch possible in find_table_ind or  we just read
unsync cashed value in the thread because st.c never used atomics.

Unfortunately I am not well familiar with Ruby threads so it is hard for
me to say how to fix it.  I only think that we should keep st.c
thread-unaware as it always was.

[normalperson (Eric Wong)]

Vladimir Makarov [e-mail] wrote:

On 02/06/2018 02:38 PM, Eric Wong wrote:

Vladimir Makarov [e-mail] wrote:

On 02/06/2018 05:00 AM, Eric Wong wrote:

during rebuild. Disabling the free(tab->entries) at line
st.c:792 (patch below) seems to indicate success with the
thread_safe test suite (letting it loop overnight).
It still crashed after four runs :< It might run longer with
the simplecov/coveralls stuff commented out in spec_helper.rb
since coverage creates a giant hash and might increase the chance
of failure.

Eric, thank you for working on the problem and analyzing it. I'll look at
this and try to fix it as soon as possible.

I reproduced this crash although the reproducing is not stable with or
without valgrind.

It is a typical data race.  The same problem existed in the old hash
tables.  It also rebuilt tables and freed old data structure.

Ah, thank you for your investigation. I missed this earlier;
but I see a problem with the uncommon calls to .eql? and .hash
via rb_any_cmp and obj_any_hash functions which may invoke
scheduling in the middle of a lookup.

File st.c was never thread-safe.  The data races are/were possible in
many places.

We could make st.c thread-safe.  But I don't think it is a right way.  It is
not a trivial task and it also will hurt performance considerably.  We still
needs thread-unaware level to work with hash tables (st.c) for cases when
tables are used internally in one thread.

100% agreed. st.c performance cannot be compromised in the
common case for thread-safety.

So I think the crash should be fixed in other places where calls of st.c
happen.

I don't know how it should be fixed because I don't know Ruby thread
semantics.  Does Ruby guarantee that there are no data races or should a
ruby programmer still provides thread synchronization despite GIL?  If it is
later, thread_safe gem is probably buggy because one thread reading a table
and another thread inserting elements while process table in a Ruby block. 
If there is no sync it is a typical data race and the result is
unpredictable.  In this case it is a segfault crash.  We could just give a
better message about the data races if segfault happens in st.c.

Yes, thread_safe does not live up to its name, apparently.

Regardless of its bugginess, I don't believe segfaults should
be triggerable from pure Ruby code.

(I do not speak for the rest of the team)

Also I don't know how GIL works.  Where the thread switching can happen.  Is
the switch possible in find_table_ind or  we just read unsync cashed value
in the thread because st.c never used atomics.

Unfortunately I am not well familiar with Ruby threads so it is hard for me
to say how to fix it.  I only think that we should keep st.c thread-unaware
as it always was.

I agree st.c should be thread-unaware.

One option is to disable thread scheduling during .eql? and
.hash calls; not sure how complex that would be (basically
reintroducing Thread.exclusive, which was recursive).

Perhaps some form of deferred reclamation (either GC or
Userspace-RCU/QSBR/EBR) can be employed during uncommon
rebuilds...

Will think about it; but I won't have much time for Ruby in
the next month or so.

[Eregon (Benoit Daloze)]

The GIL guarantees all C code is executed with the GIL held, so C code always perfectly sees effects performed by C code of other threads
(except rb_thread_call_without_gvl but it is not used here, is it?).

In this case: is the code during a Hash lookup, after calling Ruby's #hash/#eql? expecting some state to not have changed since before the call?
If so, it should be enough to re-check that state after the call.

When growing/shrinking a Hash table, the GIL should be held, so that should be observed atomically by other threads.
Or is it the problem that #rehash needs to call back to Ruby code?

[normalperson (Eric Wong)]

[e-mail] wrote:

The GIL guarantees all C code is executed with the GIL held, so C code always perfectly sees effects performed by C code of other threads
(except rb_thread_call_without_gvl but it is not used here, is it?).

Correct, GVL protects common cases (string/fixnum/etc)

In this case: is the code during a Hash lookup, after calling Ruby's #hash/#eql? expecting some state to not have changed since before the call?
If so, it should be enough to re-check that state after the call.

Yes, probably doable. I thought about this while eating;
but we can probably use rebuilds_num as a seqlock:

unsigned int seq;

retry:
seq = tab->rebuilds_num; /* needs barriers /
...
...->compare(a, b); / may rebuild */
if (tab->rebuilds_num != seq)
goto retry;

Not free in terms of overhead, but should still be cheap and
shouldn't need atomics on the reader side; only memory barriers
to ensure correct ordering with some CPUs.

When growing/shrinking a Hash table, the GIL should be held, so that should be observed atomically by other threads.
Or is it the problem that #rehash needs to call back to Ruby code?

Right, the rebuild is done entirely with the GVL held, so that's
not a problem. It needs to update rebuild_num atomically
for the above reader pseudocode to work, but that's a cold path.

Vladimir: thoughts?

[vmakarov (Vladimir Makarov)]

On 02/07/2018 06:19 PM, Eric Wong wrote:

[e-mail] wrote:

The GIL guarantees all C code is executed with the GIL held, so C code always perfectly sees effects performed by C code of other threads
(except rb_thread_call_without_gvl but it is not used here, is it?).
Correct, GVL protects common cases (string/fixnum/etc)

In this case: is the code during a Hash lookup, after calling Ruby's #hash/#eql? expecting some state to not have changed since before the call?
If so, it should be enough to re-check that state after the call.
Yes, probably doable. I thought about this while eating;
but we can probably use rebuilds_num as a seqlock:

unsigned int seq;

retry:

seq = tab->rebuilds_num; /* needs barriers /
...
...->compare(a, b); / may rebuild */
if (tab->rebuilds_num != seq)
goto retry;

Not free in terms of overhead, but should still be cheap and
shouldn't need atomics on the reader side; only memory barriers
to ensure correct ordering with some CPUs.

When growing/shrinking a Hash table, the GIL should be held, so that should be observed atomically by other threads.
Or is it the problem that #rehash needs to call back to Ruby code?
Right, the rebuild is done entirely with the GVL held, so that's
not a problem. It needs to update rebuild_num atomically
for the above reader pseudocode to work, but that's a cold path.

Vladimir: thoughts?

Yes, I think it could work. It is probably a solution with minimal
possible overhead. Besides compare, there are other external functions
called by st.c (e.g. func in st_insert2 or st_general_foreach) but
probably they don't go from C land to Ruby land (where the thread
switch can happen and the table modified or even eql method would be
written to modify the table which would be a very weird code). As I saw
these functions are used to inform GC only. But it worth to add
additional asserts to check it.

Strange that the old hash tables had no such code already. Probably we
were lucky that the bug did not triggered before.

I'll work on it. I think the patch will be ready on the next week. I
need to evaluate a performance impact too.

[vmakarov (Vladimir Makarov)]

• File thread-table-rebuild.patch thread-table-rebuild.patch added

vmakarov (Vladimir Makarov) wrote:

I'll work on it. I think the patch will be ready on the next week. I
need to evaluate a performance impact too.

The following patch solves the problem of rebuilding table in one
thread during a search of the same table in another thread. The
thread switch can happen in Ruby method implementing equal or hash and
the other thread rebuilds the table (by adding elements).

Analogous problem can happen even in one thread program if a developer
implements an equal method which can change a table (although it would
be a very weird program).

This problem results in ICE for new hash tables. For old hash
tables, the problem would have corrupted hash table structure which
probably results in later incorrect table behaviour.

The patch takes into account that the table can be rebuilt during
hashing or comparison functions and takes a special precaution for
this situation by restarting a hash table operation (or its part)
again.

Before the patch, thread-safe gem crashed 2-3 times out of each 10
runs. After applying the patch, no crashes happen (I've tried 50
runs).

As for hash table performance, fortunately, I did not found
performance degradations. Geometric mean for hash table benchmarks
run on x86-64 (i7-4790K) by

ruby ../trunk/benchmark/driver.rb -p hash -r 8 -e trunk:: -e current::

is practically the same (about 0.3% difference).

I have no write access to Ruby repository. So please, consider the
patch for inclusion into the trunk.

I cannot find where changelog entry should go for the current trunk,
but here it is.

Mon Feb 12 10:41:18 2018 Vladimir Makarov [e-mail]

    * st.c: Add comment about table rebuilding during comparison.
    (DO_PTR_EQUAL_CHECK): New macro.
    (REBUILT_TABLE_ENTRY_IND, REBUILT_TABLE_BIN_IND): New macros.
    (find_entry, find_table_entry_ind, find_table_bin_ind): Use new
    macros.  Return the rebuild flag.
    (find_table_bin_ptr_and_reserve): Ditto.
    (st_lookup, st_get_key, st_insert, st_insert2): Retry the
    operation if the table was rebuilt.
    (st_general_delete, st_shift, st_update, st_general_foreach):
    Ditto.
    (st_rehash_linear, st_rehash_indexed): Use DO_PTR_EQUAL_CHECK.
    Return the rebuild flag.
    (st_rehash): Retry the operation if the table was rebuilt.

[normalperson (Eric Wong)]

• Backport changed from 2.3: UNKNOWN, 2.4: UNKNOWN, 2.5: UNKNOWN to 2.3: REQUIRED, 2.4: REQUIRED, 2.5: REQUIRED

[normalperson (Eric Wong)]

[e-mail] wrote:

File thread-table-rebuild.patch added

Thank you for working on this!

This problem results in ICE for new hash tables. For old hash
tables, the problem would have corrupted hash table structure which
probably results in later incorrect table behaviour.

ICE - Internal Compiler Error? Or did you mean "use-after-free"?

Yes, 2.3 and earlier st.c is affected by this, too, just seems
to hit less frequently... 2.3 maintainer(s) will need to
backport from scratch, maybe

I have no write access to Ruby repository. So please, consider the
patch for inclusion into the trunk.

Looks good to me, committed as r62396.
You can probably ask hsbt and matz for commit access.

I cannot find where changelog entry should go for the current trunk,
but here it is.

We write changelog entries in the commit message nowadays. In the
future, you can send the output of "git format-patch" since it
appears you're using git anyways and we can "git am" it.

I also wrote the following text to summarize in r62396

st.c: retry operations if rebuilt

Calling the .eql? and .hash methods during a Hash operation can
result in a thread switch or a signal handler to run: allowing
one execution context to rebuild the hash table while another is
still reading or writing the table. This results in a
use-after-free bug affecting the thread_safe-0.3.6 test suite
and likely other bugs.

This bug did not affect users of commonly keys (String, Symbol,
Fixnum) as those are optimized to avoid method dispatch
for .eql? and .hash methods.

A separate version of this change needs to be ported to Ruby 2.3.x
which had a different implementation of st.c but was affected
by the same bug.

(and copied your changelog entry below)

[vmakarov (Vladimir Makarov)]

On 02/13/2018 05:13 AM, Eric Wong wrote:

[e-mail] wrote:

File thread-table-rebuild.patch added
Thank you for working on this!

This problem results in ICE for new hash tables. For old hash
tables, the problem would have corrupted hash table structure which
probably results in later incorrect table behaviour.
ICE - Internal Compiler Error? Or did you mean "use-after-free"?
Fortunately, not internal compiler error (i wrongly used the term from
GCC development :). I meant use-after-free.
Yes, 2.3 and earlier st.c is affected by this, too, just seems
to hit less frequently... 2.3 maintainer(s) will need to
backport from scratch, maybe

I have no write access to Ruby repository. So please, consider the

>> I cannot find where changelog entry should go for the current trunk, >> but here it is. > We write changelog entries in the commit message nowadays. In the > future, you can send the output of "git format-patch" since it > appears you're using git anyways and we can "git am" it. > > I also wrote the following text to summarize in r62396 > > st.c: retry operations if rebuilt > > Calling the .eql? and .hash methods during a Hash operation can > result in a thread switch or a signal handler to run: allowing > one execution context to rebuild the hash table while another is > still reading or writing the table. This results in a > use-after-free bug affecting the thread_safe-0.3.6 test suite > and likely other bugs. > > This bug did not affect users of commonly keys (String, Symbol, > Fixnum) as those are optimized to avoid method dispatch > for .eql? and .hash methods. > > A separate version of this change needs to be ported to Ruby 2.3.x > which had a different implementation of st.c but was affected > by the same bug. > > (and copied your changelog entry below) Thank you, Eric!

[vo.x (Vit Ondruch)]

Thank you for the patch. I applied it to Ruby in Fedora and so far like 10 builds of thread_safe passed without issues. I hope I'll be able to update the Ruby package soon.

[vo.x (Vit Ondruch)]

• Status changed from Open to Closed

thread_safe build just fine with the patch applied. Closing this so the patch can be backported soon.

[naruse (Yui NARUSE)]

• Backport changed from 2.3: REQUIRED, 2.4: REQUIRED, 2.5: REQUIRED to 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONE

ruby_2_5 r62858 merged revision(s) 62396.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONE to 2.3: REQUIRED, 2.4: DONE, 2.5: DONE

ruby_2_4 r63805 merged revision(s) 62396.