Project

General

Profile

Bug #15397

Ruby process doesn't release memory back to os on CentOS

Added by lihr (辉荣 李) 8 months ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
2.5.1, 2.5.3
[ruby-core:90415]

Description

I'm not familiar with low level knowledge, so I'm not sure if this issue belongs to Ruby or CentOS, sorry for this, but it's very easy to produce the problem.

Reproducer:

ruby -v, I have test this script in both 2.5.1 and 2.5.3, I installed ruby via rvm

  1. ruby test_gc.rb it will create a 50M json file and exit
  2. open 4 windows and runruby test_gc.rb in each window with delay 30 seconds to 60 seconds, each one will run about 30 minutes this time

The script is very simple(there're some differences between this and the attachment in log format and file size), the main part is list below:

Thread.new do
  started = Time.now.to_i
  while true
    sys_mem = (`ps -o rss= -p #{Process.pid}`.strip.to_i * 1024).to_hmsize
    objspace_mem = ObjectSpace.memsize_of_all.to_hmsize
    puts "#{(Time.now.to_i - started).to_hmtime}\tsys_mem:#{sys_mem}, objspace_mem:#{objspace_mem}"
    sleep 3
    GC.start
  end
end

# the script will create this file at the first launch
file = File.expand_path("../wiki_pages.json", __FILE__)

puts "read content from file #{File.size(file)}[#{File.size(file).to_hmsize}]"
# read the file, memory will raise about 50M
json = IO.read(file)
sleep 30

puts "parse json string to object"
# memory will raise about 350M
obj = JSON.parse(json)
sleep 30

puts "set string&object to nil, and sleep 30 min to check the memory usage."
# now set the json string and the parsed object to nil, I expect the memory will reduce 300M at least
json = nil
obj = nil

# let the thread to monitor memory, add a simple sleep to keep the thread alive
sleep 1800

Result of reproduce process

I have tested this script under macOS and Cent OS, and some .txt files are attached.

Expected result and the reason why you expect

However, I have no problem with the macOS version ruby, but the Cent OS version is not good.

As the two huge object (string&hash) are released, and the size of ObjectSpace.memsize_of_all shows these two objects are removed from the objspace, so the memory should be return back to os, which the CentOS version doesn't make. When the memory pressure raises, CentOS just kill one of the processes.


Files

test_gc.rb (2.21 KB) test_gc.rb lihr (辉荣 李), 12/11/2018 03:27 AM
centos-1.txt (4.03 KB) centos-1.txt lihr (辉荣 李), 12/11/2018 03:27 AM
centos-2.txt (4.08 KB) centos-2.txt lihr (辉荣 李), 12/11/2018 03:27 AM
centos-4.txt (1.9 KB) centos-4.txt lihr (辉荣 李), 12/11/2018 03:27 AM
centos-3.txt (3.21 KB) centos-3.txt lihr (辉荣 李), 12/11/2018 03:27 AM

Also available in: Atom PDF