Bug #9518: Objects in large arrays are leaked

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-darwin13.0]
• Backport: 1.9.3: DONTNEED, 2.0.0: DONTNEED, 2.1: DONE

a = [nil] * 131071
loop { a << Object.new; a.pop }

process RSS stays stable

a = [nil] * 131072
loop { a << Object.new; a.pop }

process RSS grows quickly and never falls

It seems to be related to this bit of code: https://github.com/github/ruby/blob/2.1/gc.c#L4764-4766

Oops, the markdown rendering messed up the code I pasted above.

a = [nil] * 131071
loop { a << Object.new; a.pop }
# process RSS stays stable

a = [nil] * 131072
loop { a << Object.new; a.pop }
# process RSS grows quickly and never falls

[ko1 (Koichi Sasada)]

Yes, you are right. WB (write barrier) strategy doesn't care such case.

How many such program?
If it is popular case, we can care about it.

[normalperson (Eric Wong)]

The yahns HTTP server uses a long-lived fdmap array to map
Fixnum(fileno) -> IO connections.
This array exists prevent GC from sweeping IOs (because IOs are watched
by Linux epoll and not markable w/o an Array to store them).

64K+ connections (array size) is attainable with yahns. The long-lived
fdmap array lifetime should not infect the short lived client
connections.

If I were to allow >=64K client connections on my public yahns instance,
clients will cause memory leaks/waste.

(yahns supports infinitely-lived connections, but in
practice, clients connections tend to be short-lived).

[normalperson (Eric Wong)]

ko1: may I commit the removal of special case for large array/hash?

• gc.c (rb_gc_writebarrier): remove special case for large hash/array

http://bogomips.org/ruby.git/patch?id=40849ff1bc881
git://80x24.org/ruby.git gc-nomagic

I strongly believe there should no magic based on size.

[ko1 (Koichi Sasada)]

(2014/03/29 15:48), [e-mail] wrote:

I strongly believe there should no magic based on size.

I need to ask tarui-san, who introduce this mechanism.

Tarui-san, could you give me a comment?

--
// SASADA Koichi at atdot dot net

[tarui (Masaya Tarui)]

Sorry for such a late response.

I agree.

At the first time, I was introduled for reducing minor GC's long time
by rescan of huge size array.
But in real case, if there are huge size array, there may be other huge data too.
No matter how it do, it have to take time.

• Status changed from Open to Closed
• % Done changed from 0 to 100

Applied in changeset r45638.

---

gc.c: drop special case for big hash/array

• gc.c (rb_gc_writebarrier): drop special case for big hash/array
  [Bug #9518]

[normalperson (Eric Wong)]

Thanks all! r45638

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: DONTNEED, 2.1: DONE

r45638 was backported into ruby_2_1 at r45818.
according to #9796