Project

General

Profile

Actions

Bug #22248

open

`Errno::NOERROR` from `connect(2)` once a non-main Ractor has more than one Ruby thread

Bug #22248: `Errno::NOERROR` from `connect(2)` once a non-main Ractor has more than one Ruby thread

Added by hornairs (Harry Brundage) about 11 hours ago. Updated about 11 hours ago.

Status:
Open
Assignee:
-
Target version:
-
[ruby-core:126427]

Description

Summary

On x86_64-linux, socket IO in a non-main Ractor fails spuriously as soon as that Ractor contains a second Ruby thread. The connecting thread raises Errno::NOERROR — errno 0 surfaced as a SystemCallError, whose message is literally Success:

Errno::NOERROR: Failed to open TCP connection to 127.0.0.1:38975
                (Success - connect(2) for "127.0.0.1" port 38975)

With a raw TCPSocket instead of Net::HTTP the VM can abort outright:

[BUG] rb_sys_fail_path_in(io_fillbuf, fd:6 ) - errno == 0

What the second thread does doesn't seem to matter. A thread that only calls sleep, one that only burns CPU, one doing pipe IO, and one doing socket IO all trigger it. Nothing else I varied matters: a Ractor with a single thread never fails, no matter how much socket work it does or how loaded the machine is, and a second thread in the main Ractor never causes it, but a second thread in a non-main Ractor does.

This bug made my CI jobs flaky in roughly 10% of builds, where the "second thread" was nothing more exotic than a stub HTTP server standing in for an upstream service alongside the client under test.

Reproduction

See this gist: https://gist.github.com/airhorns/dc676921e02272cc41f8aa15da040e45

ruby ractor_factors.rb interleave 200

Note that the interleaving is important to more reliably trigger the bug. On a shared machine the failure rate swings enormously with host load: run in blocks, the same arrangement scored 196/200 in one build and 0/200 in the next, so blocked counts are not comparable to each other and a 0 says nothing.

Architecture

I have only reproduced this on x86_64-linux. On aarch64-linux — official ruby:4.0, byte-identical revision 03b6d3f889 — roughly 1500 exchanges of the failing arrangements are clean, including 300 of the arrangement that fails 89/200 on x86, under 1- and 2-CPU quotas, with competing busy loops, and with 8 neighbour threads. arm64-darwin is clean too. So expect to need an x86_64 Linux host, ideally a loaded one.

Environment

ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [x86_64-linux]
Socket.tcp_fast_fallback = true (default; false reproduces identically)
RUBY_MAX_CPU unset (=1 does not help)
shared CI host, nproc 16, 1-minute load average 3.1-5.9 during the run

Relation to Bug #21195

Bug #21195 is the same shape — errno lost around io_internal_wait, fixed for 3.3 and 3.4 in 2025 — and the errno == 0 assertion text is identical. This is a live path in 4.0.6, reached through ordinary socket connect and read, with or without Happy Eyeballs, so I am filing it separately rather than commenting there. I could not find an existing report for the Ractor variant.

Updated by hornairs (Harry Brundage) about 11 hours ago Actions #1 [ruby-core:126428]

What it is not

  • Not Net::HTTP specific. Raw TCPSocket fails too, and is the case that aborts the VM.
  • Not "IO in a Ractor is unsupported". A single-threaded non-main Ractor drove the identical request 200 times without a failure, in the same run, in the same load window — including one arrangement that owns the listening socket and calls accept(2) inside the Ractor, and one that owns both ends of the connection.
  • Not the second thread's IO. A neighbour that never touches a file descriptor fails at 89/200.
  • Not CPU contention as such. The same CPU-burning thread placed in the main Ractor instead of the non-main one gives 0/200, in the same run.
  • Not Happy Eyeballs v2. With Socket.tcp_fast_fallback = false the pattern reproduces unchanged: 49, 49, 32, 20, 3 bad out of 66 for the five two-thread arrangements, 0 for all six others.
  • Not M:N native-thread migration — see below.
  • Not a native extension. The reproduction is pure stdlib. I first saw this through a Rust
    extension's Ruby HTTP transport, but the extension is absent from the script.
  • Not the connection failing for a real reason. The stub server is listening on the port the same
    iteration just obtained from TCPServer#addr, and connect(2) reports success while raising.

Native-thread migration is ruled out

The obvious guess is that errno, being per-native-thread, is read on a different native thread than the one that set it — Ruby threads in non-main Ractors being scheduled M:N. Migration is directly observable: /proc/thread-self resolves to <pid>/task/<tid> for the calling native thread and is readable from a non-main Ractor (Fiddle.dlopen is not — it raises Ractor::UnsafeError). gettid_migration.rb is attached in the gist. Three observations kill the theory:

  • On the x86_64 host where the bug fires constantly, migration was observed 0 of 6 rounds in every setting, with both a sleeping and a CPU-burning neighbour, in the same build as the failures above.
  • On aarch64-linux migration happens readily — 3 of 4 rounds for a two-thread non-main Ractor, tids moving e.g. 16 → 17 and 17 → 18 → 19 — and the bug never reproduces there at all.
  • RUBY_MAX_CPU=1 leaves one native thread to migrate between, so migration is impossible by construction (and observably stops on arm64, where it otherwise happens constantly) — the failure persists anyway, 9/200.

So migration and the failure are, if anything, anti-correlated. Whatever loses errno here, it is not a thread waking on a different native thread.

Actions

Also available in: PDF Atom