Feature #21962: Add deep_freeze for recursive freezing

• Status: Open
• Assignee: -
• Target version: -

[Eregon (Benoit Daloze)]

Motivation

It is common to want some data structure to be immutable, e.g. to ensure it doesn't get mutated by multiple threads.
See the first section of #21665 for a more detailed motivation, the summary is there is long-standing demand for it (back to #17145) for many reasons.
One way/workaround to do this currently is Ractor.make_shareable but its name suggests a Ractor-specific purpose.
The functionality of deeply freezing is separate from Ractor and it is generally useful.
Ractor.make_shareable almost has the right semantics and has been used for years so this proposal largely follows its semantics rather than reinventing the wheel.
The semantics also match the ice_nine gem, which further illustrates these semantics work well in practice and are expected.

Examples

It freezes deeply:

config = {
  db: { host: ENV["HOST"], ports: [5432] },
  flags: [:a, :b]
}.deep_freeze

config.frozen? # => true
config[:db].frozen? # => true
config[:db][:host].frozen? # => true
config[:db][:ports].frozen? # => true

Modules and classes are not frozen by deep_freeze:

String.deep_freeze
String.frozen? # => false

It works for cyclic references:

a = []
a << a
a.deep_freeze
a.frozen? # => true

It calls .freeze on each object:

class Stats
  def initialize(samples)
    @samples = samples
    @mean = nil
  end

  def mean
    @mean ||= @samples.sum / @samples.size.to_f
  end

  def freeze
    mean
    super
  end
end

s = Stats.new([1.0, 2.0, 3.0])
s.deep_freeze
s.mean # => 2.0, would raise FrozenError instead if overridden #freeze was not called

Background

#21665 relaunched the discussion about deep_freeze and it was discussed in a dev meeting.

@headius (Charles Nutter) has said he won't have time to look at it soon.
So I am proposing something concrete which hopefully can be approved as-is.

Design

• Proposed: Kernel#deep_freeze. Alternative: Object.deep_freeze.
  • Kernel#deep_freeze is consistent with Kernel#freeze.
  • [a, b, c].deep_freeze is nicer, more concise and more idiomatic than Object.deep_freeze([a, b, c]).
  • The proposal focuses on semantics; the exact method name and owner can be decided separately.
• The return value is the receiver.
• deep_freeze never freezes modules and classes and does not visit their internal references
  • Same as Ractor.make_shareable and IceNine.deep_freeze
  • No unexpected breakage because a class, its ancestors, their constants, etc all got frozen.
  • Very easy to explain in the documentation.
  • If people want to freeze classes/modules (rare case) they can with .freeze and have control and can decide whether to freeze the ancestors, constants, class variables, etc.
• deep_freeze calls freeze and ensures every object is frozen
  • Same as Ractor.make_shareable and IceNine.deep_freeze
  • The already well-known hook to e.g. compute lazy caches eagerly, etc
  • We still check that .freeze did freeze the object, and raise a RuntimeError if not (same behavior and error message as Ractor.make_shareable).
• Not a protocol of defining deep_freeze in many classes.
  • That does not work with circular references.
  • No need to invent a complex protocol when there is an existing working one (calling .freeze) which has proven to work well (by being used by Ractor.make_shareable).
• Handling circular references by maintaining internally a set of visited objects to avoid walking circular references multiple times
  • As done in obj_traverse_i in ractor.c.
  • No expectation for calling deep_freeze repeatedly on the same object to be fast
  • An object flag wouldn't work for this, for the case that freeze raises an exception.
• If some freeze method raises an exception, the exception is propagated.
  • Same as Ractor.make_shareable and IceNine.deep_freeze
  • Some objects visited before the exception may already be frozen, same as Ractor.make_shareable (unavoidable because we need to call freeze as we traverse the object graph).
• deep_freeze does not stop at already-frozen objects, because they may still reference unfrozen objects.
• No special treatment for shareable objects, they are frozen too.

I think this addresses all the points mentioned in the dev meeting discussion and in #21665 and in #17145.

The traversal and freezing behavior closely follows Ractor.make_shareable, without introducing new semantics.

Implementation

The implementation in CRuby would be shared with Ractor.make_shareable, ractor.c already provides rb_obj_traverse() which is a great fit to implement this.

From a high-level view the implementation works like this:

deep_freeze recursively traverses the reachable object graph from the receiver.
For each visited object:

• If it is a Class or Module, it is not frozen and its internal references are not traversed.
• Otherwise, Ruby calls .freeze on the object.
• After calling freeze, Ruby verifies that the object is frozen. If it is not frozen, an exception is raised (same behavior and error message as Ractor.make_shareable).
• The traversal keeps a visited set to avoid infinite recursion on cycles.

Non-goals

This proposal is focused on defining deep_freeze and nothing else:

• This proposal does not define a new protocol requiring classes to implement custom recursive logic.
• This proposal does not change Ractor.make_shareable.
• This proposal does not freeze classes or modules.
• This proposal does not attempt to define immutability as a broader language concept.

Compatibility and risk

• The proposal adds a new API and does not change existing freeze behavior.
• It intentionally reuses existing freeze overrides on user objects.
• It intentionally does not freeze classes/modules to avoid surprising breakage.
• Its semantics are close to existing Ractor.make_shareable, reducing implementation risk.

Relation to existing methods

• freeze: freezes only the receiver (shallow).
• deep_freeze: recursively freezes reachable objects.
• Ractor.make_shareable: recursively freezes reachable objects except if they are already shareable. Special handling to make some objects shareable.

[Eregon (Benoit Daloze)]

• Related to Feature #21665: Revisit Object#deep_freeze to support non-Ractor use cases added

[Eregon (Benoit Daloze)]

• Related to Feature #17145: Ractor-aware `Object#deep_freeze` added

[matheusrich (Matheus Richard)]

I can't remember any Ruby methods with the deep_ pattern (looks like Prism defines deep_freeze and Gem::ConfigFile has deep_transform_config_keys), this is intuitive for me (probably because of Rails).

I'd love to see it in core!

[matz (Yukihiro Matsumoto)]

Ractor.make_shareable does more than deep freezing and exists for Ractor. Its presence does not by itself justify deep_freeze in core.

In my opinion, deep freezing is a rare need in practice, and the ice_nine gem seems to cover it well.

Could you share concrete use cases where ice_nine is not sufficient and core inclusion is necessary?

Matz.

[headius (Charles Nutter)]

matz (Yukihiro Matsumoto) wrote in #note-4:

Ractor.make_shareable does more than deep freezing and exists for Ractor. Its presence does not by itself justify deep_freeze in core.

But deep freezing is the most visible effect, and it has been requested for core Ruby for years, maybe more than a decade now.

Ractor provides make_shareable to ensure objects can be safely used in parallel across Ractors without copying. The exact same justification applies to safely using objects across Threads, which are much more commonly used and provide parallel execution on jruby and truffleruby.

To me it makes perfect sense that this freezing capability be made general purpose rather than only available when Ractor is defined.

You also say it doesn't make sense to put it in core, but presumably Ractor will be part of core in the future. Given that everyone agrees that deep freezing is the safest way to make an object graph parallel-safe, why not move that capability to a non-Ractor location right now?

An additional reason: If jruby and truffleruby want to provide make_shareable, We need to define the Ractor namespace. But that indicates to user code that defined?(Ractor) is true, even if we don't implement the rest of Ractor's features. We are in a very difficult position because we want to provide the deep freezing capability but may not fully support Ractors for some time. What would you suggest we do?

In my opinion, deep freezing is a rare need in practice, and the ice_nine gem seems to cover it well.

Deep freezing with make_shareable is essentially the only way to share objects across Ractors, which means this capability is going to be used very heavily in the future. Currently, in order to use this feature with threads, you must call Ractor.make_shareable. The resulting objects are just as safe for threads. Why not make that capability a more general core feature for users that don't need Ractor to do parallel execution?

Could you share concrete use cases where ice_nine is not sufficient and core inclusion is necessary?

The concrete use cases are any place you want to pass around a graph of objects and know that it will not be mutated somewhere else. That obviously covers parallel execution use cases, but also cases where you are passing a graph of objects to a third party API and want to ensure they are not modified.

As pure Ruby in ice_nine, it's much slower than a built-in core method could be, and since it has to re-check the graph if called again, this performance difference compounds.

I would personally like to see deep freeze also set a "deep frozen" bit to avoid future traversals, in the same way as make_shareable.

I would like to turn this question around: What is the justification for keeping this feature specifically tied to Ractor when we have shown many cases where it is generally useful without Ractor? If a user has no interest in using Ractor, why do they need to call a Ractor method to get a deep frozen object graph?

[byroot (Jean Boussier)]

I don't have a strong opinion here, but one argument I could see for inclusion in core is to optimize frozen constants, e.g.

SCHEMA = [
  { type: :foo, tags: ["a", "b"] },
  { type: :bar, tags: ["c", "d"] },
  ...
].deep_freeze

Which is much nicer than:

SCHEMA = [
  { type: :foo, tags: ["a", "b"].freeze }.freeze,
  { type: :bar, tags: ["c", "d"].freeze }.freeze,
  ...
].freeze

[Eregon (Benoit Daloze)]

At the meeting @matz (Yukihiro Matsumoto) said he was positive that deep_freeze is "freeze but recursively".
More precisely:

• String.deep_freeze => freezes String like String.freeze but not anything further like Object or so.
  (This might unexpectedly freeze String in cases like FOO = { String => -> { ... }, Array => -> { ... } } but is deemed acceptable)
• Foo.new.deep_freeze => freezes that object but not class Foo.
• Fiber and Thread (brought up by @ko1 (Koichi Sasada)): .deep_freeze is the same as .freeze on them.
• Recursive for Array elements, Hash pairs, Set, Struct, Data, other collection-like types.
• Recursive for Object instance variable values.