Bug #22198: win32: heap overflow in Kernel#system

• Status: Closed
• Assignee: windows
• Target version: -
• ruby -v: ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x64-mingw-ucrt]
• Backport: 3.3: REQUIRED, 3.4: REQUIRED, 4.0: REQUIRED

[mame (Yusuke Endoh)]

system('foo.exe %A') looks past %A for a closing %, reading past the end of the string.

It does not reproduce normally, but with page heap enabled it faults:

PS C:\> gflags /p /enable ruby.exe /full
PS C:\> ruby -e "system('x' * 1024 + '.exe %A')"
-e:1: [BUG] Segmentation fault
ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x64-mingw-ucrt]

-- Control frame information -----------------------------------------------
c:0003 p:---- s:0011 e:000010 l:y b:0001 CFUNC  :system
c:0002 p:0013 s:0006 e:000005 l:n b:---- EVAL   -e:1 [FINISH]
c:0001 p:0000 s:0003 E:001620 l:y b:---- DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
-e:1:in '<main>'
-e:1:in 'system'

-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- C level backtrace information -------------------------------------------
C:\WINDOWS\SYSTEM32\ntdll.dll(NtWaitForSingleObject+0x14) [0x00007ffbd55000e4]
C:\WINDOWS\System32\KERNELBASE.dll(WaitForSingleObjectEx+0xaf) [0x00007ffbd274c11f]
C:\Users\YusukeEndoh\scoop\apps\ruby\current\bin\x64-ucrt-ruby400.dll(rb_vm_bugreport+0x237) [0x00007ffb37262227]
C:\Users\YusukeEndoh\scoop\apps\ruby\current\bin\x64-ucrt-ruby400.dll(rb_bug_for_fatal_signal+0x9b) [0x00007ffb37000beb]
C:\Users\YusukeEndoh\scoop\apps\ruby\current\bin\x64-ucrt-ruby400.dll(rb_shape_free_all+0xfb5) [0x00007ffb37191985]
C:\WINDOWS\System32\ucrtbase.dll(seh_filter_exe+0x84) [0x00007ffbd2fff2e4]
 [0x00007ff765e81080]
C:\WINDOWS\SYSTEM32\ntdll.dll(_chkstk+0x9f) [0x00007ffbd550479f]
*snip*

PR: https://github.com/ruby/ruby/pull/17929

[mame (Yusuke Endoh)]

• Status changed from Assigned to Closed