Bug #16455: coroutine ucontext uses deprecated POSIX getcontext/swapcontext/makecontext, absent in musl and uclibc

• Status: Closed
• Assignee: ioquatix (Samuel Williams)
• Target version: 3.0
• ruby -v: ruby 2.7.0p0 (2019-12-25 revision 647ee6f091)
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
• Tags: musl, patch, fiber

[luizluca (Luiz Angelo Daros de Luca)]

Hello,

While building ruby 2.7.0 for mips with musl, it fails:

linking miniruby                                                                                   /home/luizluca/prog-local/openwrt/trunk/staging_dir/toolchain-mips_24kc_gcc-8.3.0_musl/lib/gcc/mips-openwrt-linux-musl/8.3.0/../../../../mips-openwrt-linux-musl/bin/ld: cont.o: in function `coroutine_initialize_main':                                                                                cont.c:(.text+0x36c): undefined reference to `getcontext'                                          /home/luizluca/prog-local/openwrt/trunk/staging_dir/toolchain-mips_24kc_gcc-8.3.0_musl/lib/gcc/mips-openwrt-linux-musl/8.3.0/../../../../mips-openwrt-linux-musl/bin/ld: cont.o: in function `fiber_setcontext':                                                                                         cont.c:(.text+0xde6): undefined reference to `swapcontext'                                         /home/luizluca/prog-local/openwrt/trunk/staging_dir/toolchain-mips_24kc_gcc-8.3.0_musl/lib/gcc/mips-openwrt-linux-musl/8.3.0/../../../../mips-openwrt-linux-musl/bin/ld: cont.o: in function `fiber_switch':                                                                                             cont.c:(.text+0x16b8): undefined reference to `makecontext'                                        collect2: error: ld returned 1 exit status                                                         make[3]: *** [Makefile:271: miniruby] Error 1

It seems that now ruby either uses native ASM code or ucontext for coroutine.
Both are unavailable in my crossbuild. It was building and running with 2.6.x:

configure:25935: checking native coroutine implementation for mips-linux-gnu                       configure:25976: result: no

With 2.7.0:

configure:26536: checking native coroutine implementation for mips-linux-gnu                       configure:26601: result: ucontext

I tried to disable it with --without-coroutine, --with-out-coroutine and --with-coroutine=no, but it is innocuous or try to use a "no" implementation.

It looks like this introduced the change https://github.com/ruby/ruby/commit/7291fef55c90b9ab6b3c22018b16972861b98c9d

And musl might never fix it:
https://wiki.musl-libc.org/open-issues.html

[mame (Yusuke Endoh)]

• Assignee set to ioquatix (Samuel Williams)

[nobu (Nobuyoshi Nakada)]

luizluca (Luiz Angelo Daros de Luca) wrote:

I tried to disable it with --without-coroutine, --with-out-coroutine and --with-coroutine=no, but it is innocuous or try to use a "no" implementation.

Does —-with-coroutine=copy work?

[luizluca (Luiz Angelo Daros de Luca)]

nobu (Nobuyoshi Nakada) wrote:

luizluca (Luiz Angelo Daros de Luca) wrote:

I tried to disable it with --without-coroutine, --with-out-coroutine and --with-coroutine=no, but it is innocuous or try to use a "no" implementation.

Does —-with-coroutine=copy work?

Yes. And it also affects uclibc.

autoconf should test for getcontext/swapcontext/makecontext at link time as musl do have them declared (but not implemented). uclibc fails even before, while building as it does not even declare them.

For those cases, copy might work.

[luizluca (Luiz Angelo Daros de Luca)]

• Subject changed from coroutine ucontext uses deprecated POSIX getcontext/swapcontext/makecontext, absent in musl to coroutine ucontext uses deprecated POSIX getcontext/swapcontext/makecontext, absent in musl and uclibc

nobu (Nobuyoshi Nakada) wrote:

luizluca (Luiz Angelo Daros de Luca) wrote:

I tried to disable it with --without-coroutine, --with-out-coroutine and --with-coroutine=no, but it is innocuous or try to use a "no" implementation.

Does —-with-coroutine=copy work?

I created a patch to fix the build with musl/uclibc:

--- a/configure.ac
+++ b/configure.ac
@@ -2344,7 +2344,10 @@ AS_CASE([$rb_cv_coroutine], [yes|''], [
             rb_cv_coroutine=copy
         ],
         [*], [
-            rb_cv_coroutine=ucontext
+            AC_CHECK_FUNCS([getcontext swapcontext makecontext],
+                [rb_cv_coroutine=ucontext],
+                [rb_cv_coroutine=copy; break]
+            )
         ]
     )
     AC_MSG_RESULT(${rb_cv_coroutine})

It is ugly as it outputs AC_CHECK_FUNCS tests in the middle of checking coroutine implementations.

[luizluca (Luiz Angelo Daros de Luca)]

There is also a missing include for copy to work with musl:

+--- a/coroutine/copy/Context.h
++++ b/coroutine/copy/Context.h
+@@ -13,6 +13,7 @@
+ #include <string.h>
+ #include <stdlib.h>
+ #include <alloca.h>
++#include <sys/types.h>
+ 
+ #define COROUTINE __attribute__((noreturn)) void
+

[ioquatix (Samuel Williams)]

Copy implementation is the right one. But what architecture are you compiling for? because copy coroutine is so slow.

[ioquatix (Samuel Williams)]

• Status changed from Open to Closed
• Target version set to 36

It is fixed: https://github.com/ruby/ruby/pull/2995

[ncopa (Natanael Copa)]

This is still an issue for alpine s390x. See #16809

[puchuu (Andrew Aladjev)]

ioquatix (Samuel Williams) wrote in #note-7:

It is fixed: https://github.com/ruby/ruby/pull/2995

Hello, you have provided good fix but it is related to armv7 only.
For example: aarch64-gentoo-linux-musl and aarch64_be-gentoo-linux-musl are valid CHOSTS, but first one works fine, second one fails.
First one is using "rb_cv_coroutine=arm64", second one is using default "rb_cv_coroutine=ucontext".
Same thing will be true for s390x, other arches: it will try to use default broken (for musl) "rb_cv_coroutine=ucontext".

@luizluca (Luiz Angelo Daros de Luca) have provided right fix - it works perfect. https://github.com/ruby/ruby/pull/3567

[ioquatix (Samuel Williams)]

Understood. The PR looks acceptable and the right fix.

My advice is to avoid the copy coroutine implementation if at all possible, so we should try to detect as many native implementation paths as possible.

[hsbt (Hiroshi SHIBATA)]

• Target version changed from 36 to 3.0

[puchuu (Andrew Aladjev)]

• File aarch64_be.patch aarch64_be.patch added

I've checked that current "aarch64" coroutine works fine for "aarch64_be" (at least for qemu). So I am going to propose to update "aarch64" wildcard.

[luizluca (Luiz Angelo Daros de Luca)]

I still the get same issue with 3.0.0.

Ruby cannot blindly try to use ucontext without checking for getcontext swapcontext makecontext presence.
https://github.com/ruby/ruby/pull/3567 does work. It only needs to be refreshed, removing the #include.

[hsbt (Hiroshi SHIBATA)]

• Tags changed from patch, fiber to patch, fiber, musl