Bug #13846: Openbsd possible memory leak when using Thread

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 2.3.4
• Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN

[parhs (Coding Gorilla)]

On openBSD 5.9/6.1 running this code causes RSS memory to increase without limit. This does not happen in ubuntu or other linux.

loop do
  thr = Thread.new { puts "thread test" }
  thr.join
  GC.start
end

This is related to other things too, like when calling Open3.* , virtually everything that is using threads...
I have attached a screenshot with RSS memory plot I did.

[parhs (Coding Gorilla)]

• Description updated (diff)

[jeremyevans0 (Jeremy Evans)]

I've confirmed this also happens with the current version of OpenBSD and ruby 2.4.1. I'll have to write a C version to see if this is a problem with OpenBSD's thread library. If not, it's probably an issue with ruby. Hopefully I'll have time to work on that tonight.

[parhs (Coding Gorilla)]

I installed 2.5.0-dev (from github) and seems that this bug does not occur or at least it is fixed. I believe that it might be related to #13772. My daemons started with low memory around 30mb and ended up after a week to 300mb with high cpu usage and whole system lagging. Even local ssh connection had noticeable lag.

With 2.5.0-dev that I tested RSS memory for this example seems stable. For my daemon it is increasing but very slowly compared to how it used to though it might be normal. (I got 100kb max rss increase after 6 hours compared to 4-5mb before)

[kernigh (George Koehler)]

I can reproduce this bug in 2.3 and 2.4 but not in trunk. I am using 2.3 and 2.4 from OpenBSD packages. I compiled trunk from source.

$ dmesg | head -2
OpenBSD 6.1 (GENERIC.MP) #19: Thu Aug  3 14:59:44 CEST 2017
    [e-mail]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
$ ruby -v
ruby 2.5.0dev (2017-09-06 trunk 59764) [x86_64-openbsd6.1]
$ ruby24 -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-openbsd]
$ ruby23 -v              
ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-openbsd]

I run the loop in trunk ruby for some minutes, then watch the RES in top(1). It stays at 11M:

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
16837 kernigh   64    0   11M   11M onproc/0  -         6:50 94.73% ruby

When I run the loop in ruby23 or ruby24, I can see RES increasing slowly from 11M to 12M, 13M, and so on. The bug might be in Ruby 2.3 and 2.4, but not in Ruby trunk and not in OpenBSD's thread library.

[jeremyevans0 (Jeremy Evans)]

• Status changed from Open to Closed

I did some testing of this recently. With ruby 2.6, the memory use is constant. With ruby 2.5, there is slight memory growth. However, considering the extensive thread changes between ruby 2.5 and ruby 2.6, it is unlikely the particular fix could be isolated and backported. I'm going to close this now. If you are experiencing similar memory leaks with threads with master or 2.6, please update this ticket.