Project

General

Profile

Bug #11572

Urnary operator causing references to unreachable objects in 2.1.x ?

Added by tdg5 (Danny Guinther) about 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
[ruby-core:<unknown>]

Description

Perhaps this is an error on my part, but I stumbled across some weird GC behavior related to the unary & (ampersand) operator on 2.1.x.

I don't have any leads as to what the cause of the issue might be, but the gist of the issue is that using & with Array#each or Array#map seems to cause references to unreachable objects to be maintained, preventing those unreferenced objects from being GC'd.

The majority of my testing has been on Ubuntu 14.04.3, though a colleague was kind enough to verify that the behavior also occurs on OSX.

This seems like it is likely related to https://github.com/ruby/ruby/pull/592 which was ultimately solved by commit 2f3b28c682fe3010ed3b8803199616c12b52512d:

+Sat Apr 12 22:11:10 2014 Nobuyoshi Nakada nobu@ruby-lang.org
+

  • * string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear
  • caller's self which is useless, so that it can get collected.
  • [Fixes GH-592]

As far as I can tell, this commit was not backported to 2.1.x. If this commit did fix the issue, should it be backported to 2.1? I haven't seen been able to find an existing bug for this issue if one exists, so it's unclear to me why this wouldn't have been backported.

I've been using the script below to experiment with the phenomenon.
I also made a gist of the script here: https://gist.github.com/tdg5/0b9f145edb5114a2dca1

# Create some special classes to facilitate tracking allocated objects.
class TrackedArray < Array; end
class TrackedString < String; end
STRANG = "a" * 5000

class ClingyObjects
  def generate(should_cling = false)
    strs = TrackedArray.new
    30000.times { strs << TrackedString.new(STRANG) }

    char_count = 0
    # I'm not sure why, but using the unary & operator on the Array, whether
    # through #each or #map, prevents the allocated objects from being GC'd.
    # Maybe I'm missing something, but after this method returns nothing
    # should refer to the strs Array or any of the objects contained in the
    # Array, so GC should proceed without issue. What gives?

    strs.each(&:length) if should_cling
    strs.each {|x| char_count += x.length }
    char_count
  end

  # Helper to print object allocation stats.
  def object_stats(tag)
    puts "#{tag}:"
    puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
    puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
  end

  def print_with_stats(char_count)
    object_stats("Before GC")
    # Run the garbage collector.
    GC.start
    object_stats("After GC")
    puts char_count
  end
end

def wrapper
  clinger = ClingyObjects.new
  puts "Non-clingy:"
  count = clinger.generate
  clinger.print_with_stats(count)
  puts "\nClingy:"
  count = clinger.generate(:should_cling)
  clinger.print_with_stats(count)
  # Try to GC again for fun
  puts "\nTry GC again"
  GC.start
  clinger.print_with_stats(count)
  puts "\nDitch clinger and try GC again"
  clinger = nil
  5.times do
    GC.start
    puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
    puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
    puts "\nSleep a bit and try again"
    sleep 3
  end
  puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
  puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"
end
wrapper
puts "TrackedArray: #{ObjectSpace.each_object(TrackedArray).count}"
puts "TrackedString: #{ObjectSpace.each_object(TrackedString).count}"

Output from 1.9.3-p551, 2.1.2, 2.1.3, 2.1.5, 2.1.7:

# Non-clingy:
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 0
# TrackedString: 0
# 150000000

# Clingy:
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 1
# TrackedString: 30000
# 150000000

# Try GC again
# Before GC:
# TrackedArray: 1
# TrackedString: 30000
# After GC:
# TrackedArray: 1
# TrackedString: 30000
# 150000000

# Ditch clinger and try GC again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

# Sleep a bit and try again
# TrackedArray: 1
# TrackedString: 30000

Output from 2.2.0 (expected output):

Non-clingy:
Before GC:
TrackedArray: 1
TrackedString: 30000
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Clingy:
Before GC:
TrackedArray: 1
TrackedString: 30000
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Try GC again
Before GC:
TrackedArray: 0
TrackedString: 0
After GC:
TrackedArray: 0
TrackedString: 0
150000000

Ditch clinger and try GC again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0

Sleep a bit and try again
TrackedArray: 0
TrackedString: 0
TrackedArray: 0
TrackedString: 0

Thanks in advance!

Associated revisions

Revision a6b6f9bb
Added by usa (Usaku NAKAMURA) about 4 years ago

merge revision(s) 45576: [Backport #11572]

    * string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear
      caller's self which is useless, so that it can get collected.
      [Fixes GH-592]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_1@52362 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 52362
Added by usa (Usaku NAKAMURA) about 4 years ago

merge revision(s) 45576: [Backport #11572]

* string.c (sym_to_proc), proc.c (rb_block_clear_env_self): clear
  caller's self which is useless, so that it can get collected.
  [Fixes GH-592]

History

Updated by halogenandtoast (Matthew Mongeau) about 4 years ago

  • Tracker changed from Bug to Backport
  • Project changed from Ruby master to Backport21

I can confirm that this happens in the latest 2.1.x tag and that applying 2f3b28c682fe3010ed3b8803199616c12b52512d was able to fix this issue.

Updated by naruse (Yui NARUSE) about 4 years ago

  • Tracker changed from Backport to Bug
  • Project changed from Backport21 to Ruby master
  • Status changed from Open to Closed
  • ruby -v set to ruby 2.1.7p400 (2015-08-18 revision 51632) [x86_64-linux]
  • Backport set to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONTNEED

Updated by usa (Usaku NAKAMURA) about 4 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONTNEED to 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONTNEED

ruby_2_1 r52362 merged revision(s) 45576.

Also available in: Atom PDF