Misc #18420: Question about how to handle IO.pipe reader and writer when forking

• Status: Open
• Assignee: -

[estolfo (Emily S)]

I don't think this is a bug but I'm not sure. It could be just a limitation of forking with a IO.pipe reader and writer.

I work on the Elastic Ruby APM agent (https://github.com/elastic/apm-agent-ruby). The Ruby agent is a singleton and uses a IO.pipe to send data to the APM server. The writer we wrap in a GzipWriter. When the parent process is forked, the child inherits the readers and (gzip)writers. I detect the forking by comparing PIDs when an event is created and create new readers and writers in the child process so that I don't interfere with the parent's streams.
When the fork exits, I close the child's readers and writers but I see the follow error

zlib(finalizer): Zlib::GzipWriter object must be closed explicitly.
zlib(finalizer): the stream was freed prematurely.

because the parent's writer is still considered living in the child process.
One option to address this error is to create a finalizer on the writer that closes it. But then I get intermittent errors from the APM server saying the gzipped data's header is invalid. I'm assuming this is because the parent's writer is closed by the child before the compression is complete in the parent.

Is there a proper way to handle forking when the parent is using a pipe? Both options I have to handle the writer result in an error/warning: I can't close the GzipWriter in the child because of the corrupted data issue but if I don't close it, I get the "Zlib::GzipWriter object must be closed explicitly" warning.

If I close the reader and writer in a "before_fork" hook run in the parent, I avoid both issues. But I don't want to recreate the reader/writer objects every time the parent is forked.

I've been testing with ruby 2.7.0 and Resque (https://github.com/resque/resque) primarily but this issue is observed with Puma and other frameworks that use forking.