Bug #14471
closedDRb mixing up function return values between PIDs after fork()
Description
This is likely a duplicate of https://bugs.ruby-lang.org/issues/2718, although the Backport191 project in redmine appears to be dead.
We have found the exact same problem with modern rubies such as 2.3, 2.4, 2.5.
The summary is that due to fork and sharing of established connections in the DRbConn
pool, the DRb
server sometimes sends DRb
client 1's message to client 2 and client 2's message to client 1.
My colleague Adam Grare and I have narrowed it down to the "global" DRb::DRbConn
class instance variable @pool
being inherited in child forked processes.
Our workaround[1] was to close all existing connections in the inherited pool in each fork to ensure each child fork process would get brand new connections:
DRb::DRbConn.instance_variable_get(:@mutex).synchronize do
DRb::DRbConn.instance_variable_get(:@pool).each(&:close)
end
The attached client and server and log demonstrates the problem and fails on all modern ruby versions. The source of these can be found in a gist here: [2]. Note, we have tested this on Mac OSX and debian.
It would be great if connection pools would not be shared in forked processes or give us a public interface to clear the connection pool.
[1] https://github.com/ManageIQ/manageiq/pull/16953
[2] https://gist.github.com/agrare/d9484884bd297b1615814128129cfc5c
Files
Updated by jeremyevans0 (Jeremy Evans) over 5 years ago
- Is duplicate of Bug #2718: DRb mixing up function return values between PIDs after fork() added
Updated by jeremyevans (Jeremy Evans) about 5 years ago
- Status changed from Open to Closed
Applied in changeset git|d0ed935d5bf8c3fce9800742a36e44fb7f63dda4.
Fix some DRb issues (#2552)
- Handle BasicObject in drb
Also fix a bug in rescue clause of any_to_s because sprintf
does not handle the %l modifier.
Fixes [Bug #7833]
- Do not send a reply to the client if there is a connection error
This allows for normal TCP shutdown (fin-ack-fin-ack instead of
fin-ack-push-rst).
Patch from pierre@mouraf.org (Pierre-Alexandre Meyer).
Fixes [Bug #2339]
- Detect fork and do not reuse forked connections in drb
This associates each DRbConn with a pid, and if the pid changes,
it closes any DRbConns in the pool with a pid that no longer
matches. This fixes DRb servers from sending messages intended
for one client to another client after forking.