Bug #18388: IO.copy_stream incompatibility between Ruby 2 and Ruby 3

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 3.1.0dev (2021-12-04T07:20:30Z hack 4a6ca12904) [x86_64-darwin19]
• Backport: 2.6: DONTNEED, 2.7: DONE, 3.0: DONE

[yaojingguo (Jingguo Yao)]

Put test.rb, src and dst in the same directory:

test.rb file:

src = "src"
dst = "dst"
File.open(dst, "ab", 0644) do |dst|
  File.open(src, 'rb', 0644) do |src|
    puts "src size: #{src.size()}, dst size: #{dst.size()}"
    count = IO.copy_stream(src, dst)
    puts "count: #{count}"
  end
end

src file (file size is 3):

789

dst file (file size is 1):

2

Run test.rb with Ruby 2.6, the content of the resulted dst is:

2789

So the content of src is appended to dst with Ruby 2.6. In other words,
Open mode "a" is honored.

Run test.rb with Ruby 3, the content of the resulted dst is:

278

This behavior is different from Ruby 2.6.

The cause of the problem is IO.copy_stream uses fcopyfile function for Ruby 3.
If the following code from io.c is commented out, Ruby 3 has the same behavior
as Ruby 2.6.

#ifdef HAVE_FCOPYFILE
    ret = nogvl_fcopyfile(stp);
    if (ret != 0)
        goto finish; /* error or success */
#endif

copyfile:

int fcopyfile(int from, int to, copyfile_state_t state, copyfile_flags_t flags);

fcopyfile appends src to to and then truncates to to it's original size.
The following code shows this behavior:

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <copyfile.h>

int main() {
	copyfile_state_t state;
	state = copyfile_state_alloc();

	int to;
	to = open("to", O_APPEND | O_WRONLY, 0644);	
	if (to < 0) {
		perror("open to");
		return 1;
	}

	int from;
	from = open("from", O_RDONLY);
	if (from < 0) {
		perror("open from");
		return 1;
	}

	int ret;
	ret = fcopyfile(from, to, state, COPYFILE_DATA);
	if (ret != 0) {
		perror("fcopyfile");
		return 1;
	}

	off_t copied;
	ret = copyfile_state_get(state, COPYFILE_STATE_COPIED, &copied);
	if (ret != 0) {
		perror("copyfile_state_get");
		return 1;
	}
	printf("copied: %lld\n", copied);

	copyfile_state_free(state);
}

The following table summarizes the results after running the above code:

from	to	to after fcopyfile
789	2	278
1	2	2
1	234	2

If this problem should be fixed, I am willing to do it provided with some
guidance from the community.

Here is the detailed version information:

• Ruby 2.6 version:

$ ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]

• Ruby 3 version

ruby 3.1.0dev (2021-12-04T07:20:30Z master 4a6ca12904) [x86_64-darwin19]

• OS: macOS 10.15.5

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset git|b555e659c4974acc423083b71b1bd5ec6a926046.

---

Do not use fcopyfile if appending to non-empty file [Bug #18388]

fcopyfile appends src to to and then truncates to to it's
original size.

[nobu (Nobuyoshi Nakada)]

• Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN to 2.6: DONTNEED, 2.7: REQUIRED, 3.0: REQUIRED

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.6: DONTNEED, 2.7: REQUIRED, 3.0: REQUIRED to 2.6: DONTNEED, 2.7: REQUIRED, 3.0: DONE

ruby_3_0 f4f0c793f6eb427b0a85445bff49fdc6b73447ae merged revision(s) b555e659c4974acc423083b71b1bd5ec6a926046.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.6: DONTNEED, 2.7: REQUIRED, 3.0: DONE to 2.6: DONTNEED, 2.7: DONE, 3.0: DONE