Misc #21143: Speficy order of execution const_added vs inherited

• Status: Feedback
• Assignee: -

[fxn (Xavier Noria)]

The hooks const_added and inherited may need to be executed "together".

For example, consider:

module M
  def self.const_added(cname) = ...

  class C
    def self.inherited(subclass) = ...
  end

  class D < C; end
end

When D is defined, two hooks are set to run, but in which order?

Both orders make sense in a way:

1. When inherited is called, you can observe that the subclass has a permanent name, ergo it was assigned to a constant, which must me stored in a class or module object. Therefore, the constant was added to said class/module before inherited was invoked.

2. When const_added is called, you can const_get the symbol and observe the object is a class, hence with a superclass, hence inheritance already happened.

The patch in ruby#12759 documents and adds a test for (1). Rationale:

1. I believe it would be nice to specify this order.

2. Chose (1) because it is how it works today.

While the motivation for the patch was formal (to remove an ambiguity), after reflecting about this I realized users of Zeitwerk may depend on this. Nowadays, Zeitwerk uses const_added to set autoloads for child constants in namespaces. Thanks to the current order, code can be used in inherited hooks normally (it would not be ready if the order was different).

[fxn (Xavier Noria)]

• Description updated (diff)

[fxn (Xavier Noria)]

• Description updated (diff)

[Eregon (Benoit Daloze)]

For classes/modules not using the class/module keyword (e.g. C = Class.new(parent)), inherited must run first since the inheritance happens before the possible later assignment.

So maybe for consistency inherited should run first?
I'm unsure consistency matters too much here.

I checked and TruffleRuby currently behaves like CRuby, i.e. const_added before inherited for class/module cases.

[Eregon (Benoit Daloze)]

FWIW looking at this stuff I noticed rb_class_new() doesn't call inherited at all, but rb_define_class*() do.

[fxn (Xavier Noria)]

For classes/modules not using the class/module keyword (e.g. C = Class.new(parent)), inherited must run first since the inheritance happens before the possible later assignment.

Yes, that is why the proposed docs specifically mention the class keyword, because it is there where the ambiguity lies.

If you do two things in series, it is to be expected that their side-effects happen in series. Although when that is done in one line, the programmer needs to understand that the expression is evaluated first, and then the result assigned to the constant.

[matz (Yukihiro Matsumoto)]

• Status changed from Open to Feedback

I think @eregon's opinion here is reasonable. But I don't want to break any existing code. I'd like to experiment inherited-then-const_added order works (or not).
If it works, I'd like to change the order. If it doesn't I'like to keep the current order (and document somewhere).

Matz.

[fxn (Xavier Noria)]

In theory, there could be an incompatibility in projects using Zeitwerk.

Let's imagine you have a class C defined in c.rb and C::X defined in c/x.rb. If the parent class of C has an inherited hook, today that hook can refer to C::X.

The reason for that is that when C gets created, const_added is invoked and the loader is on time to define an autoload for :X in C. That is how you are able to refer to any constant path in your project any time. (Time ago, this was done with a trace point for the :class event.)

If we change the order, when the inherited hook is invoked C::X will raise NameError.

However, let me also say that I don't know if this usage is present out there. Referring to nested constants in your just created subclass seems a rare use case that in pure Ruby may not even be possible (using "ordinary" code, let's say).

I'd be 👍 to go with @Eregon (Benoit Daloze) suggestion because it is simpler and uniform.

[byroot (Jean Boussier)]

Calling inherited before const_added is doable and I doubt it would cause much backward compatibility issues except for one thing: the class being anonymous when inherited is called.

There are a number of inherited methods in Rails that do expect the class to have a name (it handles anonymous classes in a degraded way). Just to give one example: https://github.com/rails/rails/blob/3626b8f52c425c2dd7a0d0aa698cff9f10d789b3/actionpack/lib/action_controller/railties/helpers.rb#L12

Here module_parents rely on Module#name.

But if we still assign the name before calling inherited, then I think we can change the order without problem.

[byroot (Jean Boussier)]

Here's a tentative implementation, with some ruby specs added: https://github.com/ruby/ruby/pull/12927

[fxn (Xavier Noria)]

But if we still assign the name before calling inherited, then I think we can change the order without problem.

Just to clarify in the thread, it is more than the name. Nowadays, the constant exists in inherited (maybe your patch preserves this too?)

Provided both internal inheritance and constant assignment have happened, we are discussing the order in which the user-facing callbacks are invoked.

As the description says, in const_added, today inheritance already happened (the value has a superclass), but note that the inherited callback was not fired. This is not inconsistent if we consider internal operations to be decoupled from the public hooks. On the other hand, in inherited, the constant was already assigned (const_get gives it to you and the subclass has a permanent name).

I am fine either way, though reordering would make me document in Zeitwerk that you cannot refer to certain constants in inherited hooks. Sometimes, the need to write such docs is a signal that the feature is not quite right.

So, fine either way, but slightly slanted towards keeping things as they are and applying my doc + specs patch :).

[byroot (Jean Boussier)]

Nowadays, the constant exists in inherited (maybe your patch preserves this too?)

Right. That is something that can also have an impact. Right now my patch doesn't preserve this, but it would be possible to do it.

[Eregon (Benoit Daloze)]

@byroot (Jean Boussier) 's PR swaps rb_class_inherited() and rb_const_set(), so AFAIK the constant is not set yet in the "lexical parent/namespace" module when inherited is called.
It's kind of nice to keep "actual constant assignment" and "calling const_added" together, and also it's consistent with C = Class.new(parent) where it happens in the same order as in that PR.

I'm just not sure if it might cause compatibility issues, but I suggest to try, we are early in the dev cycle and can always revert if it causes problems.

[byroot (Jean Boussier)]

I'm just not sure if it might cause compatibility issues, but I suggest to try, we are early in the dev cycle and can always revert if it causes problems.

I feat it might cause issues, as mentioned by @fxn but yes we can try. I should add a spec for that too though.

[fxn (Xavier Noria)]

The problem with C = Class.new(parent) is that it is a totally different mindset.

In the thing we are discussing, the interpreter is in charge.

In the other example, the user has decided to decouple both things. That anonymous class could be stored in a variable, passed three frames up and down, and perhaps be assigned to a constant 10 minutes later. Or never.

That is, in my mind instantiating an anonymous class object totally decouples the callbacks in a way that is expected.

Furthermore, if you have an autoload for :C:

autoload :C, 'c'

const_added is going to be called before inherited anyway (because it is triggered by the autoload call itself). Analogous situation in my view, the user decided to decouple the actions.

[nobu (Nobuyoshi Nakada)]

byroot (Jean Boussier) wrote in #note-9:

Here's a tentative implementation, with some ruby specs added: https://github.com/ruby/ruby/pull/12927

vm_declare_class calls rb_class_inherited too.
Sorry, missed that part.

[Eregon (Benoit Daloze)]

• Related to Bug #21193: Inherited callback returns `nil` for `Object.const_source_location` added