Project

General

Profile

Actions

Bug #9518

closed

Objects in large arrays are leaked

Added by Anonymous about 10 years ago. Updated almost 10 years ago.

Status:
Closed
Assignee:
-
Target version:
-
ruby -v:
ruby 2.1.0p0 (2013-12-25 revision 44422) [x86_64-darwin13.0]
[ruby-core:60699]

Description

a = [nil] * 131071
loop { a << Object.new; a.pop }

process RSS stays stable

a = [nil] * 131072
loop { a << Object.new; a.pop }

process RSS grows quickly and never falls

It seems to be related to this bit of code: https://github.com/github/ruby/blob/2.1/gc.c#L4764-4766

Updated by Anonymous about 10 years ago

Oops, the markdown rendering messed up the code I pasted above.

a = [nil] * 131071
loop { a << Object.new; a.pop }
# process RSS stays stable

a = [nil] * 131072
loop { a << Object.new; a.pop }
# process RSS grows quickly and never falls

Updated by ko1 (Koichi Sasada) about 10 years ago

Yes, you are right. WB (write barrier) strategy doesn't care such case.

How many such program?
If it is popular case, we can care about it.

Updated by normalperson (Eric Wong) about 10 years ago

The yahns HTTP server uses a long-lived fdmap array to map
Fixnum(fileno) -> IO connections.
This array exists prevent GC from sweeping IOs (because IOs are watched
by Linux epoll and not markable w/o an Array to store them).

64K+ connections (array size) is attainable with yahns. The long-lived
fdmap array lifetime should not infect the short lived client
connections.

If I were to allow >=64K client connections on my public yahns instance,
clients will cause memory leaks/waste.

(yahns supports infinitely-lived connections, but in
practice, clients connections tend to be short-lived).

Updated by normalperson (Eric Wong) almost 10 years ago

ko1: may I commit the removal of special case for large array/hash?

  • gc.c (rb_gc_writebarrier): remove special case for large hash/array

http://bogomips.org/ruby.git/patch?id=40849ff1bc881
git://80x24.org/ruby.git gc-nomagic

I strongly believe there should no magic based on size.

Updated by ko1 (Koichi Sasada) almost 10 years ago

(2014/03/29 15:48), wrote:

I strongly believe there should no magic based on size.

I need to ask tarui-san, who introduce this mechanism.

Tarui-san, could you give me a comment?

--
// SASADA Koichi at atdot dot net

Updated by tarui (Masaya Tarui) almost 10 years ago

Sorry for such a late response.

I agree.

At the first time, I was introduled for reducing minor GC's long time
by rescan of huge size array.
But in real case, if there are huge size array, there may be other huge data too.
No matter how it do, it have to take time.

Actions #7

Updated by Anonymous almost 10 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Applied in changeset r45638.


gc.c: drop special case for big hash/array

  • gc.c (rb_gc_writebarrier): drop special case for big hash/array
    [Bug #9518]

Updated by normalperson (Eric Wong) almost 10 years ago

Thanks all! r45638

Updated by nagachika (Tomoyuki Chikanaga) almost 10 years ago

  • Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN, 2.1: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: DONTNEED, 2.1: DONE

r45638 was backported into ruby_2_1 at r45818.
according to #9796

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0