Bug #22284: Passing an object with to_int to curry as its arity causes problems with LLP64.

• Status: Open
• Assignee: -
• Target version: -
• ruby -v: master
• Backport: 3.3: REQUIRED, 3.4: REQUIRED, 4.0: REQUIRED

[YO4 (Yoshinao Muramatsu)]

The proc_curry function in proc.c is affected by differences in behavior caused by the int size in FIX2INT.
As a result, depending on the arguments, curry may not function correctly on Windows.

method(:puts).curry(1r).call(42)
=> #<Proc:0x0000000005e06c58 (lambda)> (Windows)
=> 42 (others)

The same thing happens in gems/src/rbs/test/stdlib/Proc_test.rb:113.
It also triggers an assertion in a debug build on windows.

irb(main):001> method(:puts).curry(1r).call(42)
S:\git\ruby\ruby\include\ruby/internal/arithmetic/long.h:192: Assertion Failed: rbimpl_fix2long_by_shift:RB_FIXNUM_P(x)
...(snip)
-- C level backtrace information -------------------------------------------
C:\WINDOWS\SYSTEM32\ntdll.dll(NtWaitForSingleObject+0x14) [0x00007FF82F2E0404]
C:\WINDOWS\System32\KERNELBASE.dll(WaitForSingleObjectEx+0xaf) [0x00007FF82C53C11F]
S:\git\ruby\mswin64\x64-vcruntime140-ruby410.dll(rb_print_backtrace+0x3e) [0x00007FFF45D706CA] S:\git\ruby\ruby\vm_dump.c:1137
S:\git\ruby\mswin64\x64-vcruntime140-ruby410.dll(rb_vm_bugreport+0x288) [0x00007FFF45D70958] S:\git\ruby\ruby\vm_dump.c:1482
S:\git\ruby\mswin64\x64-vcruntime140-ruby410.dll(rb_assert_failure_detail+0xbb) [0x00007FFF45C05557] S:\git\ruby\ruby\error.c:1227
S:\git\ruby\mswin64\x64-vcruntime140-ruby410.dll(rb_assert_failure+0x12) [0x00007FFF45C0549A] S:\git\ruby\ruby\error.c:1202
S:\git\ruby\mswin64\x64-vcruntime140-ruby410.dll(proc_curry+0xfa) [0x00007FFF45D48426] S:\git\ruby\ruby\proc.c:4614
...(snip)

proc.c:4614 has this

        sarity = FIX2INT(arity);

FIX2INT is being used, and based on what I've read in doc/extension.rdoc,
I'm not entirely sure if it's correct for curry to use to_int.
So just reporting this for now before submitting a PR.

[nobu (Nobuyoshi Nakada)]

• Backport changed from 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN to 3.3: REQUIRED, 3.4: REQUIRED, 4.0: REQUIRED

[YO4 (Yoshinao Muramatsu)]

If we want to correct Windows' behavior, the following would be potential patches.

--- a/proc.c
+++ b/proc.c
@@ -4611,10 +4611,11 @@ proc_curry(int argc, const VALUE *argv, VALUE self)
         arity = INT2FIX(min_arity);
     }
     else {
-        sarity = FIX2INT(arity);
+        sarity = FIXNUM_P(arity) ? FIX2INT(arity) : NUM2INT(arity);
         if (rb_proc_lambda_p(self)) {
             rb_check_arity(sarity, min_arity, max_arity);
         }
+        arity = INT2FIX(sarity);
     }

     return make_curry_proc(self, rb_ary_new(), arity);

This includes a change that limits the evaluation of to_int to a single time.