Project

General

Profile

Actions

Bug #18388

closed

IO.copy_stream incompatibility between Ruby 2 and Ruby 3

Added by yaojingguo (Jingguo Yao) 10 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 3.1.0dev (2021-12-04T07:20:30Z hack 4a6ca12904) [x86_64-darwin19]
[ruby-core:106493]

Description

Put test.rb, src and dst in the same directory:

test.rb file:

src = "src"
dst = "dst"
File.open(dst, "ab", 0644) do |dst|
  File.open(src, 'rb', 0644) do |src|
    puts "src size: #{src.size()}, dst size: #{dst.size()}"
    count = IO.copy_stream(src, dst)
    puts "count: #{count}"
  end
end

src file (file size is 3):

789

dst file (file size is 1):

2

Run test.rb with Ruby 2.6, the content of the resulted dst is:

2789

So the content of src is appended to dst with Ruby 2.6. In other words,
Open mode "a" is honored.

Run test.rb with Ruby 3, the content of the resulted dst is:

278

This behavior is different from Ruby 2.6.

The cause of the problem is IO.copy_stream uses fcopyfile function for Ruby 3.
If the following code from io.c is commented out, Ruby 3 has the same behavior
as Ruby 2.6.

#ifdef HAVE_FCOPYFILE
    ret = nogvl_fcopyfile(stp);
    if (ret != 0)
        goto finish; /* error or success */
#endif

copyfile:

int fcopyfile(int from, int to, copyfile_state_t state, copyfile_flags_t flags);

fcopyfile appends src to to and then truncates to to it's original size.
The following code shows this behavior:

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <copyfile.h>

int main() {
	copyfile_state_t state;
	state = copyfile_state_alloc();

	int to;
	to = open("to", O_APPEND | O_WRONLY, 0644);	
	if (to < 0) {
		perror("open to");
		return 1;
	}

	int from;
	from = open("from", O_RDONLY);
	if (from < 0) {
		perror("open from");
		return 1;
	}

	int ret;
	ret = fcopyfile(from, to, state, COPYFILE_DATA);
	if (ret != 0) {
		perror("fcopyfile");
		return 1;
	}

	off_t copied;
	ret = copyfile_state_get(state, COPYFILE_STATE_COPIED, &copied);
	if (ret != 0) {
		perror("copyfile_state_get");
		return 1;
	}
	printf("copied: %lld\n", copied);

	copyfile_state_free(state);
}

The following table summarizes the results after running the above code:

from to to after fcopyfile
789 2 278
1 2 2
1 234 2

If this problem should be fixed, I am willing to do it provided with some
guidance from the community.

Here is the detailed version information:

  • Ruby 2.6 version:
$ ruby --version
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
  • Ruby 3 version
ruby 3.1.0dev (2021-12-04T07:20:30Z master 4a6ca12904) [x86_64-darwin19]
  • OS: macOS 10.15.5
Actions #1

Updated by nobu (Nobuyoshi Nakada) 10 months ago

  • Status changed from Open to Closed

Applied in changeset git|b555e659c4974acc423083b71b1bd5ec6a926046.


Do not use fcopyfile if appending to non-empty file [Bug #18388]

fcopyfile appends src to to and then truncates to to it's
original size.

Actions #2

Updated by nobu (Nobuyoshi Nakada) 10 months ago

  • Backport changed from 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN to 2.6: DONTNEED, 2.7: REQUIRED, 3.0: REQUIRED

Updated by nagachika (Tomoyuki Chikanaga) 7 months ago

  • Backport changed from 2.6: DONTNEED, 2.7: REQUIRED, 3.0: REQUIRED to 2.6: DONTNEED, 2.7: REQUIRED, 3.0: DONE

ruby_3_0 f4f0c793f6eb427b0a85445bff49fdc6b73447ae merged revision(s) b555e659c4974acc423083b71b1bd5ec6a926046.

Actions #4

Updated by usa (Usaku NAKAMURA) 6 months ago

  • Backport changed from 2.6: DONTNEED, 2.7: REQUIRED, 3.0: DONE to 2.6: DONTNEED, 2.7: DONE, 3.0: DONE
Actions

Also available in: Atom PDF