Bug #18613: Voluntary wanted: Some signal-related tests fail on FreeBSD 13

• Status: Feedback
• Assignee: knu (Akinori MUSHA)
• Target version: -
• Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

[mame (Yusuke Endoh)]

Some tests fail randomly on FreeBSD 13.

http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20220216T143001Z.fail.html.gz

  1) Error:
TestThread#test_signal_at_join:
Timeout::Error: execution of assert_separately expired timeout (120 sec)
pid 30743 killed by SIGABRT (signal 6) (core dumped)
| 

    /usr/home/chkbuild/chkbuild/tmp/build/20220216T143001Z/ruby/test/ruby/test_thread.rb:1390:in `test_signal_at_join'

http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20220308T023001Z.fail.html.gz

  1) Failure:
TestThreadQueue#test_thr_kill [/usr/home/chkbuild/chkbuild/tmp/build/20220308T023001Z/ruby/test/ruby/test_thread_queue.rb:175]:
only 169/250 done in 60 seconds.

http://rubyci.s3.amazonaws.com/freebsd13/ruby-master/log/20220304T163001Z.fail.html.gz

  1) Failure:
TestIO_Console#test_intr [/usr/home/chkbuild/chkbuild/tmp/build/20220304T163001Z/ruby/test/io/console/test_io_console.rb:387]:
<"25"> expected but was
<"-e:12:in `p': \e[1mexecution expired (\e[1;4mTimeout::Error\e[m\e[1m)\e[m">.

All of the tests is related to signals. I guess something around signals has changed in FreeBSD 13, but I'm unsure. The failure occurs at high frequency in CI, but rarely does on my FreeBSD shell. We need to investigate what happens.

@knu (Akinori MUSHA) is the platform maintainer for FreeBSD but he is busy. Contribution who is familiar with FreeBSD is welcome.

To make it easy to monitor the CI results, I tentatively skipped the three tests on FreeBSD https://github.com/ruby/ruby/commit/17e09f033c4d3b786672ba16d2c5d935482a2fad . If you investigate the issue on FreeBSD, please delete test/excludes/TestThread.rb and test/excludes/TestThreadQueue.rb, and remove the guard in test/io/console/test_io_console.rb.

[mame (Yusuke Endoh)]

Now FreeBSD 12 fails as well. Probably because we upgraded the machine to 12.3.

http://rubyci.s3.amazonaws.com/freebsd12/ruby-master/log/20220529T183002Z.fail.html.gz

  1) Failure:
TestThreadQueue#test_thr_kill [/usr/home/chkbuild/chkbuild/tmp/build/20220529T183002Z/ruby/test/ruby/test_thread_queue.rb:175]:
only 156/250 done in 60 seconds.

  2) Error:
TestThread#test_signal_at_join:
Timeout::Error: execution of assert_separately expired timeout (120 sec)
pid 90353 exit 1

Such noisy failures make it difficult for us to monitor and manage the CI. Since no one seems to be able to fix it, I'm now keen to stop the FreeBSD CI machines.