Bug #16265: Test (spec) failure using current MSYS2 tools, related to -fstack-protector and possibly D_FORTIFY_SOURCE=2

• Status: Closed
• Assignee: ioquatix (Samuel Williams)
• Target version: -
• ruby -v: ruby 2.7.0dev (2019-10-19T08:19:27Z master 1d435bd51a) [x64-mingw32]
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
• Tags: thread, win

[MSP-Greg (Greg L)]

GH PR https://github.com/ruby/ruby/pull/2573 was a work-around that forced using some non-current MSYS2 packages libraries for building. These libraries are only available on Appveyor because their images are out-of-date, a new MSYS2 install would not have them.

Using current MSYS2 libraries requires adding '-fstack-protector' to LDFLAGS. I also experimented with adding 'D_FORTIFY_SOURCE=2' to CFLAGS.

Ruby build/compiles with those flags added. test-all passes. But spec tests always lock up when running:

C-API Thread function rb_thread_call_without_gvl
- runs a C function with the global lock unlocked
- runs a C function with the global lock unlocked amd unlocks IO with the generic RUBY_UBF_IO

The tests are in spec/ruby/optional/capi/thread_spec.rb, as shown:

https://github.com/ruby/ruby/blob/1d435bd51aeca75f6e437bc92de2271a716a1ced/spec/ruby/optional/capi/thread_spec.rb#L124-L141

and the freeze occurs on line 137.

I've run this once in ruby-loco, twice in my ruby fork, and several times locally. In all runs, test-all passed, and spec stopped at the test above. This uses gcc 9.2.0, and the problem would occur with anyone doing a 'fresh' install of MSYS2.

Not being a c type, this is way beyond what I can fix. Happy to try patches, etc. Thanks, Greg

[MSP-Greg (Greg L)]

I bypassed the spec mentioned above:

C-API Thread function rb_thread_call_without_gvl
- runs a C function with the global lock unlocked and unlocks IO with the generic RUBY_UBF_IO

and with it bypassed, all tests/specs reliably pass on MinGW. Since most of the CI that uses gcc is done on versions older than the current MSYS2 version (9.2.0), I don't know whether it's a general problem or just Windows related.

Either way, anyone doing a fresh MSYS2 install and then building Ruby will have issues without changes to flags and the above test/spec.

[MSP-Greg (Greg L)]

PR https://github.com/ruby/ruby/pull/2718 shows this issue, and also properly updates MSYS2/MinGW gcc.

[ioquatix (Samuel Williams)]

For some reason, this also started breaking on Windows in this PR and I had to disable it https://github.com/ruby/ruby/pull/7827/files

I cannot see how anything in my PR would impact the test that was hanging.

It seems like it might relate to the method length, so I might try reworking the PR to move the pread/pwrite functionality into a separate function completely.

However, I don't see why it should matter at all.

[ioquatix (Samuel Williams)]

I'm going to see if we can fix this.

https://github.com/ruby/ruby/pull/7848

[ioquatix (Samuel Williams)]

• Assignee set to ioquatix (Samuel Williams)

[ioquatix (Samuel Williams)]

Okay, the root cause of the hang is because read in thread_spec.c is not rb_w32_read. Something is clobbering the included #define read rb_w32_read.

[ioquatix (Samuel Williams)]

@shyouhei (Shyouhei Urabe) Do you think we should add something like:

#ifdef _WIN32
#include "ruby/win32.h"
#endif

to ruby.h?

[ioquatix (Samuel Williams)]

• Status changed from Open to Closed

Merged in https://github.com/ruby/ruby/commit/6d976eb5348098a346d82065621e37925acae8b8.

[ioquatix (Samuel Williams)]

I'm not sure how the ubf_io can be used as part of the public extension API on Windows, as it's unable to interrupt non-Ruby IO methods, i.e. it only works when using rb_w32_read and rb_w32_write.

Maybe the test should use actual IO objects, and invoke the equivalent #read and #write methods rather than depending on the native wrappers.