Feature #16965
closedio.c: Unsupported copy_file_range() not detected properly on certain kernels
Description
In recent RedHat kernels (for example: RHEL 7.8 using kernel 3.10.0-1127.8.2.el7.x86_64), copy_file_range()
may return EOPNOTSUP
when Ruby attempts to call this in IO.copy_stream
on an NFS mount. A simple FileUtils.copy_file
will fail with Operation not supported - copy_file_range
on these kernels.
This was possibly changed during a recent security release: https://access.redhat.com/errata/RHSA-2020:1465
Ruby's io.c detects whether copy_file_range()
is defined, not whether it is actually supported. The following test program illustrates the hole in the detection mechanism:
#include <syscall.h>
#include <stdio.h>
#if defined __linux__ && defined __NR_copy_file_range
# define USE_COPY_FILE_RANGE 1
#else
# define USE_COPY_FILE_RANGE 0
#endif
int main()
{
printf("copy_file_range? %d\n", USE_COPY_FILE_RANGE);
}
USE_COPY_FILE_RANGE
gets set to 1 even in when the system call doesn't succeed.
I suggest a few improvements:
- Use a compile-time test to verify that
copy_file_range()
can actually be executed. - Make it possible to disable
USE_COPY_FILE_RANGE
via a build option. Since the test in 1 could still pass if it is run on a Docker host that supportscopy_file_range()
, it would be helpful for us to manually disable it.
Reported by GitLab customers: https://gitlab.com/gitlab-org/gitlab/-/issues/218999
Updated by vo.x (Vit Ondruch) over 4 years ago
This is tracked here:
https://bugzilla.redhat.com/show_bug.cgi?id=1840284
and considered Kernel issue. I think that in recent Rubies, there is compile-time test already.
Updated by stanhu (Stan Hu) over 4 years ago
Thanks for the link! As far as I can tell, the test in https://github.com/ruby/ruby/blob/41a4c80d284a24360d3a6c5b6e5ca408ccca6efc/io.c#L11007 is still the same as I described above.
Updated by mame (Yusuke Endoh) over 4 years ago
- Assignee set to Glass_saga (Masaki Matsushita)
I think that Ruby handles the ENOSYS appropriately and fall back to fcopyfile and sendfile: https://github.com/ruby/ruby/blob/41a4c80d284a24360d3a6c5b6e5ca408ccca6efc/io.c#L11084
"Operation not supported" error is not ENOSYS but EOPNOTSUP, which Ruby does not handle. The issue is that the recent RedHat kernel may return EOPNOTSUP. The RedHat developers are aware of the incompatibility, and looks like they will change the return value to ENOSYS: https://bugzilla.redhat.com/show_bug.cgi?id=1783554
So, please wait for their fix.
Aside from that, is it good for Ruby to rescue EOPNOTSUP for future robustness? @Glass_saga (Masaki Matsushita)
Updated by mame (Yusuke Endoh) over 4 years ago
- Tracker changed from Bug to Feature
- ruby -v deleted (
ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]) - Backport deleted (
2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN)
Updated by vo.x (Vit Ondruch) over 4 years ago
I don't think EOPNOTSUP is documented error state, therefore I am against this. Not mentioning that it would need backport to older Rubies and what not. In the time this is going to be implemented in Ruby, updated RHEL kernel will be long released.
Updated by Glass_saga (Masaki Matsushita) about 4 years ago
- Status changed from Open to Closed
Applied in changeset git|93df3010482ad52e5ada2e416c996005da956e1e.
IO.copy_stream: handle ENOTSUP on copy_file_range(2)
fallback to other methods on ENOTSUP.
some RedHat kernels may return ENOTSUP on an NFS mount.
[Feature #16965]