Bug #9709: Large string causes SEGV with x64-mingw32

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.2.0dev (2014-04-07 trunk 45529) [x64-mingw32]
• Backport: 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: DONE

[h.shirosaki (Hiroshi Shirosaki)]

Creating large string causes SEGV with x64-mingw32 on Windows.

test.rb

A = ""
1000000.times do |i|
  A << "a" * 100000
end

gdb backtrace of ./miniruby test.rb

Program received signal SIGSEGV, Segmentation fault.
0x000007fefe88120b in msvcrt!memmove () from C:\Windows\system32\msvcrt.dll
(gdb) bt
#0  0x000007fefe88120b in msvcrt!memmove () from C:\Windows\system32\msvcrt.dll
#1  0x000000000054e404 in str_buf_cat (str=str@entry=115691040, ptr=ptr@entry=0x7b510e0 'a' <repeats 200 times>...,
    len=len@entry=100000) at ../../../ruby/string.c:2042
#2  0x000000000054e90a in rb_enc_cr_str_buf_cat (str=str@entry=115691040, ptr=0x7b510e0 'a' <repeats 200 times>...,
    len=100000, ptr_encindex=<optimized out>, ptr_cr=ptr_cr@entry=1048576, ptr_cr_ret=0x22eb10,
    ptr_cr_ret@entry=0x22eaf0) at ../../../ruby/string.c:2164
#3  0x0000000000553c6c in rb_str_buf_append (str=115691040, str2=115660360) at ../../../ruby/string.c:2207
#4  0x0000000000553d9f in rb_str_append (str2=115660360, str=115691040) at ../../../ruby/string.c:2220
#5  rb_str_concat (str1=115691040, str2=115660360) at ../../../ruby/string.c:2256
#6  0x00000000005ac743 in vm_exec_core (th=0x768ce00, th@entry=0x0, initial=initial@entry=0)
    at ../../../ruby/insns.def:1824
#7  0x00000000005ad661 in vm_exec (th=0x0) at ../../../ruby/vm.c:1328
#8  0x0000000000000000 in ?? ()

capa setting looks wrong in the following code. Here is a patch.

diff --git a/string.c b/string.c
index 511374c..8abfc25 100644
--- a/string.c
+++ b/string.c
@@ -2029,7 +2029,7 @@ str_buf_cat(VALUE str, const char *ptr, long len)
     if (capa <= total) {
        while (total > capa) {
            if (capa + termlen >= LONG_MAX / 2) {
-               capa = (total + 4095) / 4096;
+               capa = LONG_MAX - termlen;
                break;
            }
            capa = (capa + termlen) * 2;

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed
• % Done changed from 0 to 100

Applied in changeset r45534.

---

string.c: fix capacity

• string.c (str_buf_cat): should round up the capacity by 4KiB,
  but not number of rooms. [ruby-core:61886] [Bug #9709]

[nobu (Nobuyoshi Nakada)]

• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED

[usa (Usaku NAKAMURA)]

• Backport changed from 1.9.3: REQUIRED, 2.0.0: REQUIRED, 2.1: REQUIRED to 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: REQUIRED

Backported into ruby_2_0_0 at r46154.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: REQUIRED to 1.9.3: REQUIRED, 2.0.0: DONE, 2.1: DONE

r45534 was backported into ruby_2_1 branch at r46187.

[zeha (Christian Hofstaedtler)]

Hi,

I'm seeking explanation if this [security issue] only applies to mingw, and if so, why, as the same code appears to run on other platforms as well.

Thank you,
Christian

[nobu (Nobuyoshi Nakada)]

It's not a security issue but affects other platforms.
You need to make a 1GiB string on 32-bit platforms or on Windows, and a 4EiB string on other 64-bit platforms.