Bug #21944: "Cannot allocate memory" with M:N threads or Ractors on a low RAM Linux machine

• Status: Closed
• Assignee: ractor
• Target version: -
• Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: DONE

[jhawthorn (John Hawthorn)]

Linux default overcommit policy (vm.overcommit_memory=0) will fail for allocations which are "obvious overcommits" (https://www.kernel.org/doc/html/latest/mm/overcommit-accounting.html). Under M:N threading (including under Ractors) we allocate a ~512MB section of RAM to use (the physical pages are lazily mapped in when used). This caused the following exception on a machine with <512 MB of RAM.

Thread#initialize': can't create Thread: Cannot allocate memory (ThreadError)

This can be solved by initially mapping the region of memory in as protected, and only enabling read/write the first time a stack is used. We already added a guard page the first time a stack was used, so this just inverts that mprotect call (it also makes two calls instead of one, but since these are only used the first time a stack is used, I think that simplicity is worth the extra syscall).

https://github.com/ruby/ruby/pull/16232

[jhawthorn (John Hawthorn)]

• Status changed from Open to Closed

Applied in changeset git|407dd02c1b52b05ba55a179554b29a14e44a4b82.

---

Map M:N thread stack chunks initially as PROT_NONE

Previously we initially mapped the full 512MB chunk as
PROT_READ|PROD_WRITE and then set a guard page to PROT_NONE the first
time a new thread stack is needed. Usually that's okay as we don't touch
that memory until it is needed and so it doesn't count towards RSS.

However, on Linux even with vm.overcommit_memory=0 (the default) if on a
system (like a tiny cloud VM) with <512MB of RAM+swap that would error
with.

Thread#initialize': can't create Thread: Cannot allocate memory (ThreadError)

This changes the chunk to be mapped initially with PROT_NONE, then
instead of mapping the guard pages we map in the machine and VM stacks
using mprotect. This ensures we don't commit stack memory until it is
first used, and as a side benefit any stray pointers into unused stack
should segfault.

When a stack is freed/reused there is no change from the previous
behaviour, we just use madvise and leave the same regions in place.

[Bug #21944]

[jhawthorn (John Hawthorn)]

• Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: REQUIRED to 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: DONE