Misc #20739
open
Test suite does not carry over CLI options
Description
I'm not sure if anything needs to change here, but I wanted to report this in case anyone had any particular ideas. When running the ruby/ruby test suite, lots of tests call EnvUtil.invoke_ruby directly or indirectly through assert_in_out_err. That spawns a subprocess that runs Ruby and reports its status back through status, stdout, and stderr. The issue is, the subprocess does not always use the same options, which means there are lots of different ways it can be run.
For RJIT/YJIT, we have -DRJIT_FORCE_ENABLE and -DYJIT_FORCE_ENABLE, which forces the subprocess to have it enabled. Still, this isn't always used, so when running the test suite for RJIT/YJIT, a lot of tests are not actually running with those compilers.
Prism and parse.y have a similar problem. When running in the same process, it will respect the --parser option. But when running in a subprocess it will fall back to whatever the default parser is. This presents a problem for some tests that assert against RUBY_DESCRIPTION in a subprocess, as it may have +PRISM in the description.
We have tried to fix it before by adding RUN_OPTS/RUBYOPT to the subprocess options, but this causes lots of other problems. I am not sure of the correct solution, or if we should even bother fixing this. Please let me know what you think.
Updated by