Bug #17350: mongo-ruby-driver causes the segfault at program's exit

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: Ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN

[ttokutake (Tadatoshi Tokutake)]

Overview

mongo-ruby-driver 2.11.0 has caused the segfault at program's exit when connecting a certain MongoDB cluster.
I also reported this issue to mongo-ruby-driver.
https://jira.mongodb.org/browse/RUBY-2453

This is the reproduction example. Please check it out.
https://github.com/ttokutake/mongo-ruby-driver-segfault

You can see full result by using the above example, and also debug it by GDB.

Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
Fetching bson 4.11.1
Installing bson 4.11.1 with native extensions
Using bundler 1.17.2
Fetching mongo 2.11.0
Installing mongo 2.11.0
D, [2020-11-27T00:20:31.035863 #11] DEBUG -- : MONGODB | EVENT: #<TopologyOpening topology=Unknown[]>
... (snip)
D, [2020-11-27T00:20:31.145112 #11] DEBUG -- : MONGODB | [14] mongodb-sharded.cluster.local:27017 | admin.endSessions | SUCCEEDED | 0.000s
[BUG] Segmentation fault at 0x0000000000000050
ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]

-- Control frame information -----------------------------------------------
c:0001 p:---- s:0003 e:000002 (none) [FINISH]


-- Machine register context ------------------------------------------------
 RIP: 0x00007f52358f4078 RBP: 0x00007f5230b4fdd0 RSP: 0x00007f5230b4fc90
 RAX: 0x0000000000000000 RBX: 0x0000563b4b746370 RCX: 0x0000563b44ad4158
 RDX: 0x0000563b44ad4570 RDI: 0x0000563b4b8bd318 RSI: 0x0000563b44ad4020
  R8: 0x0000000000000011  R9: 0x0000000000000000 R10: 0x0000563b4a8dc050
 R11: 0x0000000000000003 R12: 0x0000000000000009 R13: 0x0000563b44b7f630
 R14: 0x0000563b46ce6ac0 R15: 0x0000563b498440d8 EFL: 0x0000000000010297

-- C level backtrace information -------------------------------------------
corrupted double-linked list
Aborted

Consideration

• https://bugs.ruby-lang.org/issues/16288 may be a similar issue.
  • a similar result.
• This commit has caused the segfault.
  • It does not happen if this line is commented out.

Ruby versions

• Ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
• Ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]

[mame (Yusuke Endoh)]

• Is duplicate of Bug #16288: Segmentation fault with finalizers, threads added

[mame (Yusuke Endoh)]

Thank you for the report. I can reproduce and investigated the issue.

I think this is the known (but unfixed) issue #16288. When Ruby process ends, it terminates all threads, run all finalizers, and then destruct everything. However, Mongo gem attempts to create a new thread in a finalizer. In fact, I've observed native_thread_create is called after rb_thread_terminate_all returned, and seen the following backtrace by rb_backtrace() with gdb.

        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/cluster.rb:422:in `block in finalize'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/session/session_pool.rb:118:in `end_sessions'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/op_msg_or_command.rb:27:in `execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:38:in `execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:26:in `do_execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:73:in `unpin_maybe'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:27:in `block in do_execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:43:in `add_error_labels'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:28:in `block (2 levels) in do_execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/response_handling.rb:87:in `add_server_diagnostics'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:29:in `block (3 levels) in do_execute'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:50:in `get_result'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/operation/shared/executable.rb:55:in `dispatch_message'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server.rb:410:in `with_connection'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:556:in `with_connection'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:357:in `check_out'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection_pool.rb:731:in `connect_connection'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection.rb:171:in `connect!'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/server/connection.rb:184:in `do_connect'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:162:in `socket'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:202:in `create_resolver'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:202:in `each'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address.rb:205:in `block in create_resolver'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address/ipv4.rb:93:in `socket'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/address/ipv4.rb:93:in `new'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/socket/tcp.rb:67:in `initialize'
        from /usr/local/bundle/gems/mongo-2.11.0/lib/mongo/socket/tcp.rb:41:in `connect!'
        from /usr/local/lib/ruby/2.6.0/timeout.rb:103:in `timeout'
        from /usr/local/lib/ruby/2.6.0/timeout.rb:83:in `block in timeout'
        from /usr/local/lib/ruby/2.6.0/timeout.rb:83:in `start'

@ko1 (Koichi Sasada) and @nobu (Nobuyoshi Nakada), what do you think?

[ttokutake (Tadatoshi Tokutake)]

Thank you for your investigation, mame-san!

mongo-ruby-driver has been fixed as below.
https://jira.mongodb.org/browse/RUBY-2453

So if you think this ticket is duplicated with https://bugs.ruby-lang.org/issues/16288,
There is no problem to close this one 🙏

Thanks again for your cooperation!

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed