Bug #21794: O_CLOEXEC is not available on Solaris 10

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: ruby 4.0.0dev (2025-12-18T07:47:43Z master 9f266ae674) +PRISM [sparc64-solaris2.10]
• Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN

[ngoto (Naohisa Goto)]

Because O_CLOEXEC is not available on Solaris 10, an error occurs when compiling box.c: "'O_CLOEXEC' undeclared (first use in this function)"

gcc -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2  -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wdeprecated-declarations -Wdiv-by-zero -Wduplicated-cond -Wimplicit-function-declaration -Wimplicit-int -Wpointer-arith -Wwrite-strings -Wold-style-definition -Wimplicit-fallthrough=0 -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-packed-bitfield-compat -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wunused-variable -Wmisleading-indentation -Wundef   -fno-strict-overflow -fvisibility=hidden -fexcess-precision=standard -DRUBY_EXPORT -fPIE -I. -I.ext/include/sparc64-solaris2.10 -I.ext/include -I../ruby.devel.ORIG/include -I../ruby.devel.ORIG -I../ruby.devel.ORIG/prism -I../ruby.devel.ORIG/enc/unicode/17.0.0   -Dmodular_gc_dir="" -D_XOPEN_SOURCE=600  -I/usr/local/64/lib/libffi-3.0.10/include -I/usr/local/64/include  -o box.o -c ../ruby.devel.ORIG/box.c
../ruby.devel.ORIG/box.c: In function 'copy_ext_file':
../ruby.devel.ORIG/box.c:634:63: error: 'O_CLOEXEC' undeclared (first use in this function); did you mean 'FD_CLOEXEC'?
     const int dst_fd = open(dst_path, O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC|bin, S_IRWXU);
                                                               ^~~~~~~~~
                                                               FD_CLOEXEC
../ruby.devel.ORIG/box.c:634:63: note: each undeclared identifier is reported only once for each function it appears in
../ruby.devel.ORIG/box.c: At top level:
cc1: warning: unrecognized command line option '-Wno-self-assign'
cc1: warning: unrecognized command line option '-Wno-parentheses-equality'
cc1: warning: unrecognized command line option '-Wno-constant-logical-operand'
cc1: warning: unrecognized command line option '-Wno-cast-function-type'
make: *** [box.o] Error 1

The following quick patch resolved the error.

diff --git a/box.c b/box.c
index 14f6acdd82..120bef3d3d 100644
--- a/box.c
+++ b/box.c
@@ -631,7 +631,25 @@ copy_ext_file(const char *src_path, const char *dst_path)
         return COPY_ERROR_SRC_STAT;
     }
 
+#ifdef O_CLOEXEC
     const int dst_fd = open(dst_path, O_WRONLY|O_CREAT|O_EXCL|O_CLOEXEC|bin, S_IRWXU);
+#else
+    int tmp_dst_fd = open(dst_path, O_WRONLY|O_CREAT|O_EXCL|bin, S_IRWXU);
+    do {
+        int dst_fd_flags;
+        if (tmp_dst_fd < 0) break;
+        dst_fd_flags = fcntl(tmp_dst_fd, F_GETFD, 0);
+        if (dst_fd_flags > 0) {
+            dst_fd_flags |= FD_CLOEXEC;
+            if (fcntl(tmp_dst_fd, F_SETFD, dst_fd_flags) == 0) break;
+            /* error */
+            close(tmp_dst_fd);
+            tmp_dst_fd = -1;
+            break;
+        }
+    } while(0);
+    const int dst_fd = tmp_dst_fd;
+#endif
     if (dst_fd < 0) {
         close(src_fd);
         return COPY_ERROR_DST_OPEN;

Note that Solaris 11 has O_CLOEXEC.

If someone argues that we no longer need to support older operating systems that lack O_CLOEXEC, I find it very difficult to counter that argument.

[nobu (Nobuyoshi Nakada)]

Can GH-15650 fix it?