Feature #8976

file-scope freeze_string directive

Added by Akira Tanaka almost 2 years ago. Updated 5 days ago.

[ruby-core:57574]
Status:Open
Priority:Normal
Assignee:-

Description

Yesterday, we had a face-to-face developer meeting.
https://bugs.ruby-lang.org/projects/ruby/wiki/DevelopersMeeting20131001Japan
Several committers attended.
matz didn't attended, though. (This means this issue is not concluded.)

We believe we found a better way to freeze static string literals for
less GC pressure.
"static string literal" is a string literal without dynamic expression.

Currently, f-suffix, "..."f, is used to freeze a string literal to avoid
String object allocation.

There are several problems for f-suffix:

  • The notation is ugly.
  • Syntax error on Ruby 2.0. We cannot use the feature in version independent libraries. So, it is difficult to deploy.
  • Need to modify for each string literal. This is cumbersome.

The new way we found is a file-scope directive as follows

# freeze_string: true

The above comment at top of a file changes semantics of
static string literals in the file.
The static string literals will be frozen and always returns same object.
(The semantics of dynamic string literals is not changed.)

This way has following benefits:

  • No ugly f-suffix.
  • No syntax error on older Ruby.
  • We need only a line for each file.

We can write version independent library using frozen static string literals as follows.

  • Use the directive at top of the file: # freeze_string: true Older Ruby ignore this as a comment.
  • Use "...".dup for strings to be modified. Older Ruby has small disadvantage: useless dup is called.

Note that the directive effects all static string literals regardless of
single quotes, double quotes, %q-string, %qq-string and here documents.
The reason that the directive is effective not only single quotes is
we want to use escape sequences such as \n in frozen string literals.

Also note that similar directive is already exist:

% ruby -w -e '
def m
end
'
-e:3: warning: mismatched indentations at 'end' with 'def' at 2
% ruby -w -e '# -- warn_indent: false --
def m
end
'

The directive, warn_indent: false, disables "mismatched indentations" warning.

nobu implemented this feature in the meeting.
Please attach the patch, nobu.


Related issues

Related to Ruby trunk - Feature #8977: String#frozen that takes advantage of the deduping Assigned 10/02/2013
Related to Ruby trunk - Feature #8579: Frozen string syntax Closed 06/29/2013
Related to Ruby trunk - Feature #11473: Immutable String literal in Ruby 3 Open
Duplicated by Ruby trunk - Feature #9278: Magic comment "immutable: string" makes "literal".freeze the default for that file Open 12/22/2013

History

#2 Updated by Sam Saffron almost 2 years ago

coupled with this I strongly feel we need a more usable way of using the deduping elsewhere.

Currently string#freeze will only affect the current string. If we had a string#frozen we could have it return a deduped frozen copy. From memory profiling the largest leak in ruby gems is strings that really should be duduped using a mechanism like it.

raised a separate issue on this: http://bugs.ruby-lang.org/issues/8977

#3 Updated by Sam Saffron almost 2 years ago

Can we also have a global switch to enable this everywhere (for debugging), it can make it simple to isolate the spots where it would fall over.

#4 Updated by Koichi Sasada almost 2 years ago

(2013/10/02 13:18), sam.saffron (Sam Saffron) wrote:

Can we also have a global switch to enable this everywhere (for debugging), it can make it simple to isolate the spots where it would fall over.

+1. It should be another ticket.

--
// SASADA Koichi at atdot dot net

#5 Updated by Akira Tanaka almost 2 years ago

akr (Akira Tanaka) wrote:

There are several problems for f-suffix:

  • The notation is ugly.

I forgot to mention Akira Matsuda's presentation at RubyShift 2013:
http://sssslide.com/speakerdeck.com/a_matsuda/rails-engines-from-the-bottom-up
(The presentation shows code snippets with f-suffix extensively.)

He said that the response of audience for f-suffix was negative.

#6 Updated by Charlie Somerville almost 2 years ago

I forgot to mention Akira Matsuda's presentation at RubyShift 2013:
http://sssslide.com/speakerdeck.com/a_matsuda/rails-engines-from-the-bottom-up
(The presentation shows code snippets with f-suffix extensively.)

He said that the response of audience for f-suffix was negative.

To be fair, you wouldn't really use f-suffix strings in all the places they were used in the presentation. They're only really useful in tight loops, or in code where aesthetics does not matter (eg. code generated from ERB).

#7 Updated by Gabriel Sobrinho almost 2 years ago

Maybe I'm too late but why not use the same object when calling String#freeze?

I mean, currently this:

"something".freeze.object_id
=> 70273877530260
"something".freeze.object_id
=> 70273877536840

And change the compiler to do this:

"something".freeze.object_id
=> 70273877530260
"something".freeze.object_id
=> 70273877530260

Not sure about the work that need to be done and even if it's possible but it would maintain the syntax compatibility with legacy versions of ruby.

I'm not against the "something"f syntax, regardless it's really strange for the ruby idiom, I'm concerned about libraries that needs to run on legacy rubies.

#8 Updated by Thomas Enebo almost 2 years ago

I think having a pragma at the top of the file will be much more error prone than the f-syntax. As a file grows, the ability to notice you are in a frozen string file goes down. It would have been great if Ruby had started immutable strings by default but that ship has sailed, I think having some files be immutable will be confusing.

Are we sure we cannot find a nicer syntax for frozen strings: %f{hello, I am frozen}?

#9 Updated by Charles Nutter almost 2 years ago

I agree with Tom here. I think it's going to be almost useless to have a full-file "freeze-string" directive.

  • From file to file, the meaning of a literal string would change. This would be confusing for everyone dealing with a project. They'd get frozen string errors in one file and not in another for exactly the same code.

  • Users would be forced to create new mutable strings with String.new. There would be no other way. So in one file you could create a new mutable string with "" and in another you'd have to use String.new.

It would be a very bad idea to have a directive that completely changes the meaning of code from one file to another.

#10 Updated by Brian Shirai almost 2 years ago

It would be a very bad idea to have a directive that completely changes the meaning of code from one file to another.

For consistency sake, it should be noted that, in fact, this is exactly what the existing encoding pragma does, and it's also the express purpose of refinements.

Hence, a more nuanced argument than this broad stroke of "very bad idea" may be needed.

#11 Updated by Charles Nutter almost 2 years ago

brixen (Brian Shirai) wrote:

For consistency sake, it should be noted that, in fact, this is exactly what the existing encoding pragma does, and it's also the express purpose of refinements.

The encoding directive changes the interpretation of the bytes within strings, but does not change their behavior. If m17n is working properly, you may never even see a difference in code, since even strings with different encodings can be negotiated into combining, matching regexp, and converting to other encodings.

Refinements change the meaning of code within a lexical scope...not within an entire file (unless it is the file's scope that is being refined). This is more analogous to instance_eval on a block, which changes what "self" methods are called against. You are correct that they do change the meaning of code within their scope, but whether that's a good feature or not is beyond the scope of this discussion. I do not particularly like refinements.

A frozen string directive would actually change the behavior of the strings in that file, making operations that worked before fail to work under the directive. Encoding does not make some methods on string start to raise errors, except where you may have differing encodings (which can happen without an encoding directive too).

Hence, a more nuanced argument than this broad stroke of "very bad idea" may be needed.

I'm not sure this is the place to have a meta-argument about how to argue for or against this proposal. But since you suggest a more nuanced argument, I suggest you look at the original points in my comment that explain why it would be a bad idea.

Do you have any arguments to make for or against this proposal?

#12 Updated by Thomas Enebo almost 2 years ago

Brian since I have been able to infer you dislike both M17n and refinements that you agree with Charlie and I that this particular pragma might not be an idea you endorse? Perhaps you can elucidate a better argument against it?

#13 Updated by Yui NARUSE almost 2 years ago

enebo (Thomas Enebo) wrote:

I think having a pragma at the top of the file will be much more error prone than the f-syntax. As a file grows, the ability to notice you are in a frozen string file goes down. It would have been great if Ruby had started immutable strings by default but that ship has sailed, I think having some files be immutable will be confusing.

Enhance your IDE.

Are we sure we cannot find a nicer syntax for frozen strings: %f{hello, I am frozen}?

Almost all of the idea to add new syntax break 2.0 compatibility, and it makes adoption of frozen strings slow.
It is the motivation of this suggestion.

#14 Updated by Martin Dürst almost 2 years ago

On 2013/10/03 2:27, brixen (Brian Shirai) wrote:

Issue #8976 has been updated by brixen (Brian Shirai).

It would be a very bad idea to have a directive that completely changes the meaning of code from one file to another.

For consistency sake, it should be noted that, in fact, this is exactly what the existing encoding pragma does,

The reason why there is an encoding pragma, and why it's per file, is
because text editors deal with one encoding per file. Doing something
like an encoding pragma e.g. on a block basis would not work well
together with editors.

I agree with Charles and others that a file-based directive isn't a good
idea for frozen/fixed strings.

From a more general perspective, it feels to me that introducing all
these frozen options will increase performance, but at the cost of
programmer effort. That would be the case also e.g. for something like
type hints,..., but that's not Ruby style.

Regards, Martin.

and it's also the express purpose of refinements.

Hence, a more nuanced argument than this broad stroke of "very bad idea" may be needed.


Feature #8976: file-scope freeze_string directive
https://bugs.ruby-lang.org/issues/8976#change-42221

Author: akr (Akira Tanaka)
Status: Open
Priority: Normal
Assignee:
Category:
Target version: current: 2.1.0

#15 Updated by Charles Nutter almost 2 years ago

duerst (Martin Dürst) wrote:

From a more general perspective, it feels to me that introducing all
these frozen options will increase performance, but at the cost of
programmer effort. That would be the case also e.g. for something like
type hints,..., but that's not Ruby style.

Personally, I think the more important benefit of having instantly-frozen literal strings, arrays, and hashes is for safer concurrency and data integrity.

  • If I pass you a reference to a string, I can create that string frozen and be sure you don't modify it. Same goes for arrays and hashes.
  • If I am initializing a global array or hash that should never be modified, I can create it frozen immediately.

Of course these can all be done by calling .freeze on the object as well, but creating immutable structures right away avoids any mistakes. And then the potential VM optimizations are a bonus on top of that.

But I stand by my opinion that a global pragma for frozen literal strings is a bad idea, because it makes all literal strings in the file start raising errors for half their methods, and it makes it impossible to copy/paste from one file to another without code potentially breaking.

#16 Updated by Robert A. Heiler almost 2 years ago

I am mildly in favour of it so +1

As it is compatible with older ruby I see little harm in it. But please don't forget proper documentation, if this is given the thumbs up by matz!

#17 Updated by Thomas Enebo almost 2 years ago

naruse (Yui NARUSE) wrote:

enebo (Thomas Enebo) wrote:

I think having a pragma at the top of the file will be much more error prone than the f-syntax. As a file grows, the ability to notice you are in a frozen string file goes down. It would have been great if Ruby had started immutable strings by default but that ship has sailed, I think having some files be immutable will be confusing.

Enhance your IDE.

It is an answer but one I think is not acceptable (obviously that is only my opinion).

Are we sure we cannot find a nicer syntax for frozen strings: %f{hello, I am frozen}?

Almost all of the idea to add new syntax break 2.0 compatibility, and it makes adoption of frozen strings slow.
It is the motivation of this suggestion.

Yeah. ko1 talked to me yesterday about this. I have been trying to think of other ways to not break backwards compatibility and have not thought of anything.

It is possible to put out a PL to 2.0 to add the syntax but obviously a decision like that is a big one. Older 2.0 libraries will not be able to read it. OTOH, MRI has been periodically putting out security fixes so perhaps a syntax error to force an upgrade is not a bad thing?

#18 Updated by Thomas Enebo almost 2 years ago

enebo (Thomas Enebo) wrote:

naruse (Yui NARUSE) wrote:

enebo (Thomas Enebo) wrote:

I think having a pragma at the top of the file will be much more error prone than the f-syntax. As a file grows, the ability to notice you are in a frozen string file goes down. It would have been great if Ruby had started immutable strings by default but that ship has sailed, I think having some files be immutable will be confusing.

Enhance your IDE.

It is an answer but one I think is not acceptable (obviously that is only my opinion).

Are we sure we cannot find a nicer syntax for frozen strings: %f{hello, I am frozen}?

Almost all of the idea to add new syntax break 2.0 compatibility, and it makes adoption of frozen strings slow.
It is the motivation of this suggestion.

Yeah. ko1 talked to me yesterday about this. I have been trying to think of other ways to not break backwards compatibility and have not thought of anything.

It is possible to put out a PL to 2.0 to add the syntax but obviously a decision like that is a big one. Older 2.0 libraries will not be able to read it. OTOH, MRI has been periodically putting out security fixes so perhaps a syntax error to force an upgrade is not a bad thing?

Read "Older 2.0 libraries will not be able to read it" as "Older MRI 2.0 implementations will not be able to read libraries which use this new syntax."

#19 Updated by Akira Tanaka almost 2 years ago

2013/10/2 enebo (Thomas Enebo) tom.enebo@gmail.com:

Issue #8976 has been updated by enebo (Thomas Enebo).

I think having a pragma at the top of the file will be much more error prone than the f-syntax. As a file grows, the ability to notice you are in a frozen string file goes down. It would have been great if Ruby had started immutable strings by default but that ship has sailed, I think having some files be immutable will be confusing.

I think it is not a big problem because
we don't need to aware the directive is exist or not if we use
"..." to strings not modified and "...".dup to strings to be modified.
--
Tanaka Akira

#20 Updated by Yusuke Endoh almost 2 years ago

"...".dup looks too verbose to me.
How about using "..." for a mutable string and '...' for an immutable?

I'm not so keen on a file-scope directive itself, though...

Yusuke Endoh mame@tsg.ne.jp

#21 Updated by Akira Tanaka almost 2 years ago

2013/10/8 mame (Yusuke Endoh) mame@tsg.ne.jp:

Issue #8976 has been updated by mame (Yusuke Endoh).

"...".dup looks too verbose to me.
How about using "..." for a mutable string and '...' for an immutable?

I considered it at first.
But we (at the meeting) abondoned it because
we want to use escape sequences, such as \n, in immutable strings.

I wrote this concern in as follows.

| Note that the directive effects all static string literals regardless of
| single quotes, double quotes, %q-string, %qq-string and here documents.
| The reason that the directive is effective not only single quotes is
| we want to use escape sequences such as \n in frozen string literals.
--
Tanaka Akira

#22 Updated by Anonymous almost 2 years ago

"..."f might be mildly ugly, but is hard to beat.
5 minutes of my thinking did not yield any better idea.
I share mame's feeling abou the file-scope directive.

#23 Updated by Charles Nutter almost 2 years ago

boris_stitnicky (Boris Stitnicky) wrote:

"..."f might be mildly ugly, but is hard to beat.
5 minutes of my thinking did not yield any better idea.
I share mame's feeling abou the file-scope directive.

I still don't like file-scope directive since it changes behavior rather than just content. In other words, a file that gains a frozen string directive suddenly creates strings that are only half functional.

See also #8992 that might address all issues by simply making the compiler and #freeze methods smarter.

  • Compiler would see through "literal".freeze and do what "literal"f" does now.
  • String#freeze could be adapted to use the fstring cache internally, so all frozen strings would be interned (in the Java sense).
  • No new backward-incompatible syntax.
  • Easy expansion to other literal syntaxes like arrays and hashes.

I think we need to kill off the "literal"f syntax and a file-scope directive is not the right way to do it.

#24 Updated by Akira Tanaka almost 2 years ago

2013/10/10 headius (Charles Nutter) headius@headius.com:

Issue #8976 has been updated by headius (Charles Nutter).

See also #8992 that might address all issues by simply making the compiler and #freeze methods smarter.

  • Compiler would see through "literal".freeze and do what "literal"f" does now.
  • String#freeze could be adapted to use the fstring cache internally, so all frozen strings would be interned (in the Java sense).
  • No new backward-incompatible syntax.
  • Easy expansion to other literal syntaxes like arrays and hashes.

#8992 doesn't address the problem follows.

| * Need to modify for each string literal.
| This is cumbersome.
--
Tanaka Akira

#25 Updated by Benoit Daloze almost 2 years ago

akr (Akira Tanaka) wrote:

2013/10/10 headius (Charles Nutter) headius@headius.com:

Issue #8976 has been updated by headius (Charles Nutter).

See also #8992 that might address all issues by simply making the compiler and #freeze methods smarter.

  • Compiler would see through "literal".freeze and do what "literal"f" does now.
  • String#freeze could be adapted to use the fstring cache internally, so all frozen strings would be interned (in the Java sense).
  • No new backward-incompatible syntax.
  • Easy expansion to other literal syntaxes like arrays and hashes.

#8992 doesn't address the problem follows.

| * Need to modify for each string literal.
| This is cumbersome.
--
Tanaka Akira

I think freezing these literals by adding ".freeze" to each of them is appropriate, these literals should already (in current version) be frozen to prevent any modification.

Changing semantics file-based is much too dangerous. And calling #dup just to have a mutable static literal String has no good meaning, it is just weird.

#26 Updated by Hiroshi SHIBATA over 1 year ago

  • Target version changed from 2.1.0 to current: 2.2.0

#27 Updated by Akira Tanaka 11 months ago

Recent Eric Wong's effort reminds me this issue.

I still think this issue is worth to consider.

Ruby 2.1 changed the semantics of "...".freeze to avoid string allocation.
It is not used extensively, I feel.

File scope directive provides a way to use frozen string literals with
much fewer modification to programs.

Also, I think frozen string literals is a better programming style.
It needs dup call ("...".dup) for string literals to be modified.
It makes us to prevent unintentional modification and
we can distinguish string literals to be modified or not, more easily.

I would like that future Ruby (Ruby 3.0?) will interpret string literals as frozen by default.
This issue can be considered as a migration path.
We can introduce file-scope directive now and change the default at future when
most programs uses frozen string literals.

#28 Updated by Eric Wong 11 months ago

akr@fsij.org wrote:

Recent Eric Wong's effort reminds me this issue.

I still think this issue is worth to consider.

Ruby 2.1 changed the semantics of "...".freeze to avoid string allocation.
It is not used extensively, I feel.

Right, "...".freeze is too ugly IMHO. Ruby should stay beautiful :)

File scope directive provides a way to use frozen string literals with
much fewer modification to programs.

Also, I think frozen string literals is a better programming style.
It needs dup call ("...".dup) for string literals to be modified.

I also think needing to call "...".dup is ugly and will break much
existing code[1].

It makes us to prevent unintentional modification and
we can distinguish string literals to be modified or not, more easily.

My concern is optimizing Ruby for the typical script language users[2],
not the (few) Rubyists who understand VM internals.

I prefer we continue to improve Ruby, transparently:

  • 2.1 (past) - { "str" => ... }, "str".freeze
  • 2.2 (done) - h"str"
  • 2.2 (planned) - .{gsub/sub/tr/tr_s}(["str"]|/../, "str"), ==, %, many more core (cfunc) methods

All of the above are transparent to existing code.

I would like that future Ruby (Ruby 3.0?) will interpret string
literals as frozen by default.
This issue can be considered as a migration path.
We can introduce file-scope directive now and change the default at
future when most programs uses frozen string literals.

I am against this; especially as a default for 3.0[1]. File scope
directives makes refactoring and reorganization of code more difficult:
move a method to a new file and behavior changes.

I hope we can implement more optimization transparently in the compiler.
For example; we should be able to optimize more cases:

    def foo(a, b)
 a.gsub(/baz/, b)
    end

    foo(a, "lit") # no dup if `a' is a string

I think we can also use use (internal) C-API changes to mark cfuncs as
taking const args. It would be useful for things like Hash#merge!, too;
not just string literals.

[1] - Even for Ruby 3.0; I strongly favor maintaining backwards
compatibility as much as possible in Ruby-land.
1.8 -> 1.9 was very painful for some users (including myself)

#29 Updated by Akira Tanaka 10 months ago

Eric Wong wrote:

Right, "...".freeze is too ugly IMHO. Ruby should stay beautiful :)

Agreed.

I also think needing to call "...".dup is ugly and will break much
existing code[1].

I don't think "...".dup is ugly.

I agree changing the default breaks much existing code.
So changing the default is long term goal.

It makes us to prevent unintentional modification and
we can distinguish string literals to be modified or not, more easily.

My concern is optimizing Ruby for the typical script language users[2],
not the (few) Rubyists who understand VM internals.

Unnecessary string duplications problem is easy to understand.
I think there is possibility to change the users.

I am against this; especially as a default for 3.0[1]. File scope
directives makes refactoring and reorganization of code more difficult:
move a method to a new file and behavior changes.

There are many context dependency for refactoring and reorganization now.
One more context dependency is not a big problem.

#30 Updated by Akira Matsuda 12 days ago

Let us bring this two-year-aged very well matured feature proposal back to the table!

I would like to propose again to put this magic comment for freezing String literals into next version of Ruby.

motivation

There are several reasons that we need this feature now:

(1) Freezing Strings is getting people's attraction recently, because people have been started realizing that they can somewhat improve their app performance by freezing Strings.
Hence, we see so many "performance improvement" patches that add .freeze to String literals here and there for frameworks, libraries, and even for ruby stdlibs.
These "performance improvement".freeze patches would certainly improve some micro-benchmark results, but at the same time inject code ugliness in exchange for the speed.
This apparently is a wrong approach. This is not a Ruby way. Ruby code must be kept beautiful.

(2) While discussing this freezing magic comment feature, Matz finally decided to make all String literals frozen by default (#11473).
Having a magic comment switch per file would be a very good transition path to the coming big change in Ruby 3.

the magic comment

We chose this at the meeting.

# frozen_string_literal: true

command line option

A command line option switch for "ruby 3 mode" will also be added.
If the option and the magic comment conflicts, the magic comment should win.

creating a String with an Encoding

While investigating ruby code in the stdlib, we found that in most cases mutating literal Strings happens in this form

"".force_encoding(Encoding::ASCII_8BIT)

so we decided to introduce a new String costructor taking Encoding via kwarg as follows:

String.new("", encoding: Encoding::ASCII_8BIT)

current status

Matz already accepted this proposal at the developer meeting in Tokyo yesterday, so this feature will be put into 2.3 unless we see any strong objection here.

#31 Updated by Jeremy Evans 12 days ago

I don't think the magic comment approach for freezing string literals is a good idea. For ruby libraries that are maintained, the maintainers will change their code so that they work when string literals are frozen by default (in ruby 3). For ruby libraries that are not maintained, the magic comment will not be added.

I am strongly in favor of freezing string literals by default in ruby 3. Most of the objections I've read against freezing string literals by default are regarding breaking of unmaintained gems. I think all ruby needs is a command line flag (e.g. --freeze-string-literals=no|yes) or environment variable (e.g. RUBY_FREEZE_STRING_LITERALS=0|1) that sets whether string literals should be frozen by default.

With the command line flag or environment variable, you can test in ruby 2.3 that your code will work in ruby 3. In ruby 3, if you are using a library/app that doesn't work with frozen string literals, you can turn it off. Turning off frozen string literals in ruby 3 will make things work, but hurt performance for the entire application, which will give people a reason to fix the code. If you add the ability for per-file magic comments, it will encourage people not to make their code work with frozen string literals, and it will make it harder to reason about code that uses string literals, since you'll have to check for a magic comment every time.

The only reason I can think of to support a magic comment for frozen string literals is if the maintainer of a library did not know how to make their library work when string literals are frozen by default. They could then use a magic comment and sweep the problem under the rug. Is that something we want to encourage?

#32 Updated by Akira Tanaka 11 days ago

Jeremy Evans wrote:

I don't think the magic comment approach for freezing string literals is a good idea. For ruby libraries that are maintained, the maintainers will change their code so that they work when string literals are frozen by default (in ruby 3). For ruby libraries that are not maintained, the magic comment will not be added.

I am strongly in favor of freezing string literals by default in ruby 3. Most of the objections I've read against freezing string literals by default are regarding breaking of unmaintained gems. I think all ruby needs is a command line flag (e.g. --freeze-string-literals=no|yes) or environment variable (e.g. RUBY_FREEZE_STRING_LITERALS=0|1) that sets whether string literals should be frozen by default.

I think chaning the default behavior without migration path is too big pain.

A command line option, --enable-frozen-string-literal, is also planned at the developper meeting DevelopersMeeting20150820Japan.
https://bugs.ruby-lang.org/projects/ruby/wiki/DevelopersMeeting20150820Japan
https://docs.google.com/document/d/1e00tTj8ix2ofS8H2RiIMUhr39bABSc7AL0vk4KbtSF4/pub

Of course, its companion, --disable-frozen-string-literal, will be available too.

With the command line flag or environment variable, you can test in ruby 2.3 that your code will work in ruby 3. In ruby 3, if you are using a library/app that doesn't work with frozen string literals, you can turn it off. Turning off frozen string literals in ruby 3 will make things work, but hurt performance for the entire application, which will give people a reason to fix the code. If you add the ability for per-file magic comments, it will encourage people not to make their code work with frozen string literals, and it will make it harder to reason about code that uses string literals, since you'll have to check for a magic comment every time.

The only reason I can think of to support a magic comment for frozen string literals is if the maintainer of a library did not know how to make their library work when string literals are frozen by default. They could then use a magic comment and sweep the problem under the rug. Is that something we want to encourage?

file-scope directive enables us to update libraries incrementally.

We can't udpate all library at once.
There are too many libraries and authors.
Some libraries will be updated soon and other libraries don't.

We can obtain performance benefit from updated libraries.
This encourage library update with smaller pain.

#33 Updated by Colin Kelley 11 days ago

Akira Tanaka wrote:

We can't update all libraries at once.
There are too many libraries and authors.
Some libraries will be updated soon and other libraries don't.

I completely concur. We will need to work through many projects and libraries and gems to make this change. The comment directive is perfect for that because we can mark files/projects/gems that have been converted and tested while not disrupting others. This is exactly how we managed the transition to utf-8 and that went very smoothly.

We can obtain performance benefit from updated libraries.

Yes, we want the benefit sooner than Ruby 3.0, and we want to make the transition over time, before 3.0 becomes mainstream. Just like with utf-8.

Also, here is a command line gem to automatically add the magic comment to the .rb files in the current folder: https://github.com/Invoca/magic_frozen_string_literal

#34 Updated by Jeremy Evans 11 days ago

Colin Kelley wrote:

Akira Tanaka wrote:

We can't update all libraries at once.
There are too many libraries and authors.
Some libraries will be updated soon and other libraries don't.

I completely concur. We will need to work through many projects and libraries and gems to make this change. The comment directive is perfect for that because we can mark files/projects/gems that have been converted and tested while not disrupting others. This is exactly how we managed the transition to utf-8 and that went very smoothly.

Adding support for this magic comment will ensure that this problem will remain a problem far in the future. We should encourage people to fix the problem, not sweep the problem under the rug.

This is not like the encoding magic comment, which is necessary because the ruby source file could be in any encoding. In general, frozen string literal issues are easy to fix, unlike encoding issues. I'd guess that 80% of exceptions caused by this change will be on the line after the string literal.

No library should be mutating a string they don't own as a side effect, and this change will make it much easier to detect libraries that do. Adding a magic comment to such a library will make it more likely that the problem isn't discovered even after ruby 3.0 is released. We should not encourage behavior that will hide bugs.

Any time spent adding magic comments to existing libraries would be better spent just making sure that libraries work with frozen string literals. Assuming we have at least 2 years between 2.3 and 3.0, that should be enough time to test gems for compatibility with --enable-frozen-string-literal before such behavior becomes the default.

It needs to be painful to use libraries that don't support frozen string literals, in order to encourage people to fix those libraries. For those people who must use libraries that still don't support frozen string literals when ruby 3 is released, they can use --disable-frozen-string-literals and be no worse off than they are now.

#35 Updated by Yura Sokolov 11 days ago

New constructor is a good thing.
But it is better to allow construction

"".b

to be mutable string. "".force_encoding(Encoding::ASCII_8BIT) is used for compatibility with Ruby 1.9.3 (cause "".b were not backported :( )

#36 Updated by _ wanabe 10 days ago

#37 Updated by Akira Tanaka 10 days ago

It needs to be painful to use libraries that don't support frozen string literals, in order to encourage people to fix those libraries. For those people who must use libraries that still don't support frozen string literals when ruby 3 is released, they can use --disable-frozen-string-literals and be no worse off than they are now.

I don't think it should be painful.
The changing default at Ruby 3.0 is big pain.
So, we should mitigate the pain.

#38 Updated by Yusuke Endoh 8 days ago

Akira Matsuda wrote:

Hence, we see so many "performance improvement" patches that add .freeze to String literals here and there for frameworks, libraries, and even for ruby stdlibs.

I'm interested in the actual effect of this feature. How many times faster will Rails run actually? Anyone measured?

https://github.com/rails/rails/commit/5bb1d4d288d019e276335465d0389fd2f5246bfd

This committer seemed to perform only micro benchmark. Was there any reason why he didn't measure the actual time?

0.4530373333993266 miliseconds saved per request

So we're shaving off 1 second of execution time for every 220 requests.

Calculation looks wrong. Correctly, 1 second per 2200 requests?

Yusuke Endoh mame@ruby-lang.org

#39 Updated by Yusuke Endoh 8 days ago

Yusuke Endoh wrote:

I'm interested in the actual effect of this feature. How many times faster will Rails run actually? Anyone measured?

I did.

Experimental overview:

  • I used akr's "immutable by default" branch
  • I executed:
    • gem install rails
    • rails new scaffold-test
    • rails s
    • open "Welcome aboard" page
    • ab -u 1000 -c 100 http://localhost:3000
  • I compared it with 2.2.1p85 (released version)
  • Whenever "can't modify frozen String" exception is raised, I created and applied a very ad-hoc fix. I didn't count the fixes precisely, but I think about 50 fixes are needed. (rdoc, bundler, and gem install sqlite3, required many fixes.)

Results: Requests per second

  • immutable by default branch: 167.14 [#/sec]
  • 2.2.1p85: 168.42 [#/sec]

I tried a few times, but I couldn't see any difference.

Note:

  • The current source of Rails already includes many .freeze literals. So there may be little room for improvement by this feature. The same experiment with removing all .freeze might be valuable.
  • I don't know how meaningful it is to measure the "requests per second" of "Welcome aboard" page on WEBrick.
  • My fixes for "can't modify frozen String" is really ad-hoc; they may break something.
  • Sorry, I didn't record my fixes, so I cannot provide the sufficient information for replication. But I could perform this experiment in a few hours, so I think it is not so difficult for other people to replicate this experiment. (I'm not familiar with Rails. Actually I used Rails for the first time in about 8 years.)

Yusuke Endoh mame@ruby-lang.org

#40 Updated by Rafael França 5 days ago

Here you can see more information about the actual effect of frozen string on Ruby on Rails https://github.com/rails/rails/pull/21057. It is 11% in a relative small application, in big applications this improvement can be bigger.

I don't know how meaningful it is to measure the "requests per second" of "Welcome aboard" page on WEBrick.

You are hitting a static page so it will not even hit the default Rails stack. You could try to hit the scaffold-test URL to get more meaningful results.

Also available in: Atom PDF