Misc #22001: Adding TruffleRuby in the CI of all default & bundled gems

• Status: Rejected
• Assignee: -

[Eregon (Benoit Daloze)]

I would like to add TruffleRuby (i.e. ruby-version: truffleruby, the latest release for improved stability) in the CI of all default & bundled gems.
I'm tracking the progress in https://github.com/truffleruby/truffleruby/issues/2644.

The reason is default & bundled gems are more likely to depend on CRuby internals or CRuby specifics, and have a very large blast readius, and so adding TruffleRuby there would help to avoid such gems breaking unknowingly on TruffleRuby, which has happened a number of times in the past (and similarly on JRuby).

The concrete issue is that for some gems without active maintainers (such as in #21922 and more) it can take a (sometimes very) long time to merge the PR adding TruffleRuby in CI.
How can we solve this?

I also don't want TruffleRuby being in CI to be any kind of burden, so if there is any issue with it more frequent than other Ruby versions in CI, it seems fair to remove it or use continue-on-error: true or just ignore the failure.
If removing, my only request would be if possible to include @eregon in the PR description removing the job so I can take a look.
To help keeping CIs green I use truffleruby-gem-tracker and I try to check it regularly to fix any failure.

[Eregon (Benoit Daloze)]

• Related to Misc #21922: Permissions for committers for ex-default/bundled/unbundled gems repositories added

[st0012 (Stan Lo)]

As discussed in person today, let's see if we can configure TruffleRuby runs in a separate yml file so we can disable the workflow when it fails without changing code. This way we can further reduce the work gem maintainers need to do to disable/re-enable TruffleRuby CI.

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Rejected

I'm opposed to adopting this as a blanket policy.

We already have a precedent where TruffleRuby CI for a library stayed broken for over a month with no one fixing it.

https://github.com/ruby/rdoc/pull/1586

Even with the ability to disable workflows from the GitHub UI (#note-2), someone still has to notice the failure and act on it — and in practice that burden falls on gem maintainers, not you.

Please don't adopt this ruby organizaion wide. Instead, reach out to each gem's maintainer individually and discuss TruffleRuby CI case by case.

[Eregon (Benoit Daloze)]

hsbt (Hiroshi SHIBATA) wrote in #note-3:

We already have a precedent where TruffleRuby CI for a library stayed broken for over a month with no one fixing it.

https://github.com/ruby/rdoc/pull/1586

For the record, I did work hard on fixing that and the initial fix took only 6 days (between report and fix) for something so major as reimplementing Ripper: https://github.com/ruby/rdoc/pull/1636
Unfortunately there was a second error, which was noticed and fixed later.
Please don't say "no one fixing it" when I worked hard on it.

I get your point though, it's bad if it stays red for any amount of time.
I wish there was a way to automatically disable if it gets red and it's caused by a breaking change in truffleruby, and not caused by a PR to the gem.
Using a truffleruby release is I think a good solution for that, as it reduces the "breaking change in truffleruby" failures to once per truffleruby release.

I try to watch failures with https://github.com/truffleruby/truffleruby-gem-tracker on a regular basis (e.g. weekly), that enables me to find gems failing if there wasn't any issue filed about it.

Please don't adopt this ruby organizaion wide. Instead, reach out to each gem's maintainer individually and discuss TruffleRuby CI case by case.

OK.