Feature #22081: Core type definition migration from `ruby/rbs` to `ruby/ruby`

• Status: Open
• Assignee: soutaro (Soutaro Matsumoto)
• Target version: 4.1

[soutaro (Soutaro Matsumoto)]

Currently, the RBS type definitions for core libraries live in the ruby/rbs repository. When you change a core class or method in ruby/ruby, the corresponding RBS type definition often needs to be updated too — but that update requires a separate PR to ruby/rbs, so the work cannot be completed within ruby/ruby alone. This is painful for Ruby core committers and contributors.

To address this problem, we will move the core library type definitions to ruby/ruby so that you can update the RBS type definitions within the same PR. We want to get this done in Ruby 4.1.

Question: Do you have any concerns about migrating the RBS core type definitions to ruby/ruby?

Current workflow

The test-bundled-gems CI job in ruby/ruby runs the core type definition tests bundled with the rbs-gem. The tests actually call core methods and validate that the arguments and return values have the expected types using is_a?.

When you change a core library in ruby/ruby in a way that requires a type definition update, the test-bundled-gems job fails and CI turns red. At that point, you have two options:

1. Update the RBS type definitions by submitting a separate PR to ruby/rbs
2. Skip the failing tests by adding the test name to the rbs_skip_tests file (in ruby/ruby repository)

Either way, you are forced to take some action to get CI green again.

Option 1 is actually very painful in practice. The type definitions bundled with the rbs-gem correspond to the released version of Ruby, not the in-development version. As a result, PRs to ruby/rbs for in-development APIs often end up modifying tests rather than type definitions — wasteful and frustrating work.

Some examples: https://github.com/ruby/rbs/pull/2366 https://github.com/ruby/rbs/pull/2706

Updated workflow

We can streamline the workflow by moving the core library RBS type definitions to ruby/ruby repository.

• We will add a directory such as sig to ruby/ruby and store files like string.rbs there
• Ruby's CI will run tests (test-bundled-gems) for the RBS type definitions in ruby/ruby repository. If your commit changes the core library methods/classes, please update the RBS files and tests too.
• Ruby 4.1 will ship with the core type definitions, which will be installed with it. The rbs-gem will load the core library type definitions shipped with Ruby.

With this workflow, when test-bundled-gems fails, you can fix the issue directly within your ruby/ruby work — without leaving the repository or coordinating a separate PR — which makes updating RBS much easier.

Note: This workflow introduces a new step — updating the RBS type definitions — that wasn't previously required of Ruby committers. However, it replaces the existing workaround of skipping tests or coordinating a separate PR to ruby/rbs just to keep CI green. And unlike that workaround, updating the RBS definitions is itself a real improvement to Ruby. We believe this trade-off is acceptable.

Implementation plan

What we won't do

1. We will keep rbs-gem as a bundled gem. We won't make it a default gem, default library, or Ruby built-in. rbs-gem has a C extension, and changing from a bundled gem would require significantly more work for compilation.
2. We will keep the ruby/rbs repository for development of the library implementation. We want to release new versions of the gem more often than Ruby's release cycle.

File layout in ruby/ruby repository

• Type definitions will be stored under sig/ (e.g., sig/core/string.rbs)
• Type definition tests will be placed under test/sig
• The bundled version of rbs-gem will be used to test against the sig/ type definitions and test/sig tests

Note that the directory names are tentative. Let me know if you have an idea what the names of the directories should be.

Installation and distribution

When Ruby is installed, the core and standard library RBS type definitions will be installed alongside it. The rbs-gem will reference these bundled definitions rather than maintaining its own copy.

The RBS files will be installed under something like "#{RbConfig::CONFIG["prefix"]}/sig", but the exact path is not decided yet. Suggestions are welcome!

Note: We may also want an install-time option to skip installing the RBS type definitions, for example for minimal Ruby builds where they are not needed.

Releasing new type definitions

Currently, we can release a new version of rbs-gem to update the core type definition. After this migration, we need to wait for the next Ruby release to update type definition.

I think this is okay, since the core type definition is pretty stable today.

Testing

RBS type definition tests will run as part of make test-all in ruby/ruby, rather than only through test-bundled-gems. This ensures type definitions are validated during the standard Ruby development workflow. Note that the RBS test suite currently takes 2–3 minutes in CI, so we need to decide which development tasks should include full testing.

Other notes

• Core RBS definitions contain documentation copied from RDoc, which needs to be updated before each release. For now, @soutaro (Soutaro Matsumoto) plans to do this manually.
• Ruby 4.0 and earlier releases do not include RBS type definitions, so the rbs-gem releases must continue to bundle them. @soutaro (Soutaro Matsumoto) will copy the type definitions from ruby/ruby to ruby/rbs at each rbs-gem release.

[nobu (Nobuyoshi Nakada)]

soutaro (Soutaro Matsumoto) wrote:

Currently, the RBS type definitions for core libraries live in the ruby/rbs repository. When you change a core class or method in ruby/ruby, the corresponding RBS type definition often needs to be updated too — but that update requires a separate PR to ruby/rbs, so the work cannot be completed within ruby/ruby alone. This is painful for Ruby core committers and contributors.

To address this problem, we will move the core library type definitions to ruby/ruby so that you can update the RBS type definitions within the same PR. We want to get this done in Ruby 4.1.

+1

The RBS files will be installed under something like "#{RbConfig::CONFIG["prefix"]}/sig", but the exact path is not decided yet. Suggestions are welcome!

I think it should be under $(datarootdir), if sig won't conflict with the other products.

Note: We may also want an install-time option to skip installing the RBS type definitions, for example for minimal Ruby builds where they are not needed.

An install-time option, instead of configure-time?

In "Updated workflow":

• Ruby's CI will run tests (test-bundled-gems) for the RBS type definitions in ruby/ruby repository. If your commit changes the core library methods/classes, please update the RBS files and tests too.

In "Testing":

RBS type definition tests will run as part of make test-all in ruby/ruby, rather than only through test-bundled-gems. This ensures type definitions are validated during the standard Ruby development workflow. Note that the RBS test suite currently takes 2–3 minutes in CI, so we need to decide which development tasks should include full testing.

Which test does it?
And I'm uncertain if bundled gems can run during test-all for now.

[ksss (Yuki Kurihara)]

The ruby/rbs repository consists of type definitions that are loaded by default (core) and those that can be loaded optionally (stdlib).

• Could you please clarify the scope of the type definitions to be migrated? For instance, are we considering only the core definitions, or both core and stdlib?
• As the core and stdlib definitions may be subject to relocation, it would be beneficial to discuss the necessary steps and procedures for such a transition.

[hsbt (Hiroshi SHIBATA)]

Currently, the RBS type definitions for core libraries live in the ruby/rbs repository. When you change a core class or method in ruby/ruby, the corresponding RBS type definition often needs to be updated too — but that update requires a separate PR to ruby/rbs, so the work cannot be completed within ruby/ruby alone. This is painful for Ruby core committers and contributors.

We already have tool/rbs_skip_tests for skipping them. I'm not sure whether forcing contributors to directly modify sig would be more effective than simply skip it to tool/rbs_skip_tests.

RBS type definition tests will run as part of make test-all in ruby/ruby, rather than only through test-bundled-gems.

I also not sure that's working correctly. I remember we exclude .so/.bundle of bundled gems from LOAD_PATH while make test-all. I suggest to separate it from test-all and added make exam.

Currently, we can release a new version of rbs-gem to update the core type definition. After this migration, we need to wait for the next Ruby release to update type definition.

It seems that new rbs versions without the bundled sig directory will become unusable in Ruby 4.0 and older versions. Is it okay?

[soutaro (Soutaro Matsumoto)]

Thanks nobu!

I think it should be under $(datarootdir), if sig won't conflict with the other products.

Sure! It would be better place for these files. I'm good if the directory is anywhere accessible through RbConfig::CONFIG or something.

An install-time option, instead of configure-time?

A configure-time option should be better. (I was not carefully distinguishing between install-time and configure-time options when I wrote that note.)

Which test does it?
And I'm uncertain if bundled gems can run during test-all for now.

I'm not sure which task is the best place for this (maybe make exam as hsbt suggests?), but I think we need a lightweight way to run the RBS tests separately from test-bundled-gems.

If running a bundled gem before installation is difficult, I think we can copy the minimum code needed to run the tests from the ruby/rbs repository into ruby/ruby (and treat that copy as read-only.)

[soutaro (Soutaro Matsumoto)]

Hi ksss,

I'm planning to migrate the RBS files under core and stdlib in ruby/rbs repo.

Currently stdlib contains some gem libraries, and we should migrate them to be bundled with each gem or distributed through gem_rbs_collection beforehand.

[Eregon (Benoit Daloze)]

This sounds great.

The workflow to update tool/rbs_skip_tests always felt very unexpected to me and I'm sure others.
The natural thing is to make a PR to ruby/rbs but that actually didn't work since ruby/rbs only has types for the latest release and not ruby master (which is very unusual for a default/bundled gem to effectively not support ruby master).

I agree core library type signatures belong inside ruby/ruby.
I think updating them will be easy (not needed for most PRs) and much nicer than just adding more and more to tool/rbs_skip_tests and explaining to everyone hitting that error the strange workflow we currently have.

What should happen for the tests for these signatures of core methods?
Just typechecking is fine, but clearly more is done currently, e.g. https://github.com/ruby/rbs/blob/master/test/stdlib/Array_test.rb
I guess that's because it's not really possible to typecheck methods written in C, except by calling them and checking they accept such arguments.

soutaro (Soutaro Matsumoto) wrote:

Ruby 4.0 and earlier releases do not include RBS type definitions, so the rbs-gem releases must continue to bundle them. @soutaro (Soutaro Matsumoto) will copy the type definitions from ruby/ruby to ruby/rbs at each rbs-gem release.

Only for rbs releases from rbs release branches for Ruby <= 4.0, right? For newer rbs gem releases they would not contain core library definitions, correct?

soutaro (Soutaro Matsumoto) wrote:

We will keep rbs-gem as a bundled gem. We won't make it a default gem, default library, or Ruby built-in. rbs-gem has a C extension, and changing from a bundled gem would require significantly more work for compilation.

Default gems can have C extensions too, so I wonder what feels as a lot of work for that.
I agree this is out of scope of this issue though, and there is no need currently to change that.
I think in some future it could be nice if Ruby itself could use the data from these rbs files.
For example we could use it to return a more meaningful Method#parameters for methods defined in C (i.e. properly representing the number of parameters and names, not just [[:rest]]), that would be really nice.
Or even a Method#signatures returning the various signatures it accepts (e.g. [[Integer], [Integer, String]]).
Anyway, not something for now.

[soutaro (Soutaro Matsumoto)]

Thanks hsbt!

We already have tool/rbs_skip_tests for skipping them. I'm not sure whether forcing contributors to directly modify sig would be more effective than simply skip it to tool/rbs_skip_tests.

I agree that this adds one more step for contributors. I cannot say it is totally free.

But skipped tests usually still need to be fixed eventually by updating the RBS files. If the RBS files live in ruby/ruby and the update is reasonably straightforward, I think it is better to make that update in the same PR as the Ruby change. The API change and the type definition change can then be reviewed together, and this workflow should make it easier to keep the RBS definition correct.

tool/rbs_skip_tests would still remain available when the RBS update is too hard or unclear.

I also not sure that's working correctly. I remember we exclude .so/.bundle of bundled gems from LOAD_PATH while make test-all. I suggest to separate it from test-all and added make exam.

That makes sense! Thanks for the suggestion.

It seems that new rbs versions without the bundled sig directory will become unusable in Ruby 4.0 and older versions. Is it okay?

We will continue shipping the core definitions bundled with the rbs-gem for Ruby 4.0 and older. I will copy the RBS files from ruby/ruby to ruby/rbs for each rbs-gem release.

[soutaro (Soutaro Matsumoto)]

Thanks, Eregon!

What should happen for the tests for these signatures of core methods?

I plan to keep the same set of tests, but move them into the ruby/ruby repository.

Since we cannot statically type check the implementation, especially the parts written in C, what we can do is call the methods and dynamically check the argument and return value types.

Only for rbs releases from rbs release branches for Ruby <= 4.0, right? For newer rbs gem releases they would not contain core library definitions, correct?

New rbs-gem releases will continue to ship with core RBS definitions. The definitions will be copied from ruby/ruby, but the bundled definitions will be used only when the Ruby installation does not provide core RBS definitions itself.

For Ruby 4.1 and later, rbs-gem will load core RBS definitions from the Ruby installation. For Ruby 4.0 and earlier, rbs-gem will load the bundled core RBS definitions.

(After a few years, we should be able to drop the core RBS definitions bundled with the rbs-gem.)

For example we could use it to return a more meaningful Method#parameters for methods defined in C (i.e. properly representing the number of parameters and names, not just [[:rest]]), that would be really nice.

That would be really great! I think this is a first step toward that kind of deeper integration between Ruby and RBS. (But, as you said, this is not what I want to discuss right now.)

[hsbt (Hiroshi SHIBATA)]

I am not sure who actually benefits from this proposal. The early explanation was:

Currently, the RBS type definitions for core libraries live in the ruby/rbs repository. When you change a core class or method in ruby/ruby, the corresponding RBS type definition often needs to be updated too — but that update requires a separate PR to ruby/rbs, so the work cannot be completed within ruby/ruby alone. This is painful for Ruby core committers and contributors.

But the later explanation became:

If the RBS files live in ruby/ruby and the update is reasonably straightforward, I think it is better to make that update in the same PR as the Ruby change. The API change and the type definition change can then be reviewed together, and this workflow should make it easier to keep the RBS definition correct.

Now, I understand this proposal as moving the maintenance cost from RBS maintainers onto every ruby/ruby contributor, by asking us to fix sig files for each core change.

If the goal is purely to make it easier for people who want to fix sig, I think we should revisit the test strategy first, before moving sig into ruby/ruby and asking everyone to update it. Concretely, we could stop testing sig files on the ruby repo side (keep only the rbs runtime test in test-bundled-gems), and then work on making sig easier to update on the rbs repo side.