https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112013-02-15T10:34:14ZRuby Issue Tracking SystemRuby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=363002013-02-15T10:34:14Zphluid61 (Matthew Kerwin)matthew@kerwin.net.au
<ul></ul><p>Note that this is closely related to <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Symbol.defined? and/or to_existing_symbol (Open)" href="https://bugs.ruby-lang.org/issues/7795">#7795</a> (Symbol.defined? and/or to_existing_symbol)</p>
<p>In existing code, Symbol.[] could be implemented as:</p>
<pre><code>class Symbol
def self.[](string)
all_symbols.find{|sym| sym.to_s == string}
end
end
</code></pre> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=363022013-02-15T10:55:12ZStudent (Nathan Zook)blogger@pierian-spring.net
<ul></ul><p>It could, but it would be extraordinarily slow, as all_symbols returns an array which is quite large in many applications.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=363052013-02-15T12:58:38Zdrbrain (Eric Hodel)drbrain@segment7.net
<ul></ul><p>To make this proposal useful all existing libraries must be updated to use the method to create symbols.</p>
<p>Other proposals such as <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a> or <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Let symbols be garbage collected (Closed)" href="https://bugs.ruby-lang.org/issues/7791">#7791</a> allow rubyists to avoid a symbol creation DoS without forcing them to ask for new releases of a library.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=363072013-02-15T13:58:55ZStudent (Nathan Zook)blogger@pierian-spring.net
<ul></ul><p>=begin<br>
Ticket <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a> requires the manipulation of global state. I'm not sure why I have to explain that this is a REALLY bad idea.</p>
<p>Ticket <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Let symbols be garbage collected (Closed)" href="https://bugs.ruby-lang.org/issues/7791">#7791</a> has two possible implementations. One is to GC symbols globally. This would require treating not just symbols like objects, but methods (whose names are in fact symbols) as well. I do not believe that methods are even currently part of the object system.</p>
<p>Another implementation would be to divide symbols into two kinds depending on how they are created. The theory being that symbols used for method names would be immune to GC. The first problem with this is that there is no reason to believe that method declarations are the first place that a particular symbol would be declared. The second is that dynamic method creation is an important part of ruby. If the goal is to protect against memory leaks in this fashion, it is not at all certain that the leak does not extend into the realm of method creation.</p>
<p>In other words, both of these implementations involve complex changes to the guts of Ruby, and lead to the likelihood of a significant behavioural fork with other rubys. (Not to mention the relatively high risk of bug introduction.) Since this is a security feature, I think that it is important to lead the way in a direction that is easy to import to other rubies (and also to backport as a security patch!) I expect Symbol[] to have a very straightforward implementation that is well-isolated from the rest of Ruby, with the possible exception of YAML.<em>load</em>, which might well benefit from such a feature.</p>
<p>As for requiring the libraries to all be updated to make use of this feature--I consider that to be a good thing. <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a> creates a change in MRI's behaviour that WILL break apparently "safe" use of existing libraries. <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Let symbols be garbage collected (Closed)" href="https://bugs.ruby-lang.org/issues/7791">#7791</a> necessarily dramatically affects Symbol's runtime performance, and thus means that any highly-tuned ruby is going to have issues--assuming that no bugs occur, and that the other rubys pick it up.</p>
<p>Furthermore, for most, perhaps even all, libraries, (({grep -R to_sym lib})) is going to tell you what you need to examine to make use of this feature. Certainly, it would be nice to avoid having to do such things, but because of the recent exploits, the more security-minded portion of the community (such as myself) is ALREADY nervously poking around in their libraries.</p>
<p>This feature gives the community a clean way to patch questionable code, which is itself relatively easy to identify in manner that makes it easy for other rubies to quickly follow. I do not believe that the other proposals do.<br>
=end</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=364762013-02-18T09:19:12Zko1 (Koichi Sasada)
<ul><li><strong>Assignee</strong> set to <i>matz (Yukihiro Matsumoto)</i></li></ul> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=373482013-03-07T12:50:44Zphluid61 (Matthew Kerwin)matthew@kerwin.net.au
<ul><li><strong>File</strong> <a href="/attachments/3581">symbol_lookup.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3581/symbol_lookup.patch">symbol_lookup.patch</a> added</li></ul><p>=begin<br>
I've attached a patch that defines ((%Symbol[str]%)). If ((|str|)) is a string and there exists a symbol such that (({symbol.to_s == str})), it returns that symbol. Otherwise it returns ((|nil|)). Raises a TypeError if ((|str|)) is not a string.</p>
<p>I also made a unit test, currently available as a gist: <a href="https://gist.github.com/phluid61/5105458" class="external">https://gist.github.com/phluid61/5105458</a><br>
=end</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=373512013-03-07T15:03:39ZStudent (Nathan Zook)blogger@pierian-spring.net
<ul></ul><p>:)</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=373562013-03-07T21:50:10Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>To obtain existing symbol, rb_check_id() is already available, so you don't have to add new extern function.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=373702013-03-08T08:48:42Zphluid61 (Matthew Kerwin)matthew@kerwin.net.au
<ul><li><strong>File</strong> <a href="/attachments/3583">symbol_lookup2.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3583/symbol_lookup2.patch">symbol_lookup2.patch</a> added</li></ul><p>nobu (Nobuyoshi Nakada) wrote:</p>
<blockquote>
<p>To obtain existing symbol, rb_check_id() is already available, so you don't have to add new extern function.</p>
</blockquote>
<p>Thank you for the feedback. With that in mind, I've made a less invasive version which only modifies string.c</p>
<p>Please let me know if my enthusiasm gets annoying. :)</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=373872013-03-08T18:36:05Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>Why does it have -1 arity?</p>
<p>And I don't think it's harmful if the method allows a Smbol too.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=374202013-03-09T17:54:38Zphluid61 (Matthew Kerwin)matthew@kerwin.net.au
<ul><li><strong>File</strong> <a href="/attachments/3587">symbol_lookup3.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3587/symbol_lookup3.patch">symbol_lookup3.patch</a> added</li><li><strong>File</strong> <a href="/attachments/3588">symbol_lookup3_warn.patch</a> <a class="icon-only icon-download" title="Download" href="/attachments/download/3588/symbol_lookup3_warn.patch">symbol_lookup3_warn.patch</a> added</li></ul><p>nobu (Nobuyoshi Nakada) wrote:</p>
<blockquote>
<p>Why does it have -1 arity?</p>
<p>And I don't think it's harmful if the method allows a Smbol too.</p>
</blockquote>
<p>To the first: an oversight on my part, there's no real reason. I have rewritten it with an arity of 1.</p>
<p>To the second: I can easily change it to allow a Symbol as well. However since the original discussion that spawned this proposal was focused on the idea of not creating unwanted/unneeded Symbols, I wonder should it emit a warning in that case?</p>
<p>I see now, too, that I was rather overzealous in my original attempts. I should have realised most of the hard work has already been done. :)</p>
<p>Now I suppose it's up to Matz to approve it or not.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=401962013-06-29T08:29:05Zphluid61 (Matthew Kerwin)matthew@kerwin.net.au
<ul></ul><p>=begin<br>
In the intervening months I've created a gem ((<a href="URL:https://rubygems.org/gems/symbol_lookup" class="external">URL:https://rubygems.org/gems/symbol_lookup</a>)) that implements ((%Symbol.[]%)), as well as two methods inspired by <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Symbol.defined? and/or to_existing_symbol (Open)" href="https://bugs.ruby-lang.org/issues/7795">#7795</a> :</p>
<ul>
<li>String#interned => gets an existing symbol, returning the symbol or nil</li>
<li>String#to_existing_sym => gets an existing symbol, raising an argument error if it doesn't exist</li>
</ul>
<p>The problem is that they are written as C extensions, and I'm not familiar enough with non-MRI implementations to port (or create a multi-platform version of) the gem; so the uptake is relatively limited. If it was promoted to core the functionality would become available to everyone, and I'm certain it would be used, for example the Rails team could use it to build an alternate solution to the problem addressed in <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a>.<br>
=end</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=403392013-07-08T04:32:25Zcabo (Carsten Bormann)cabo@tzi.org
<ul></ul><p>Let me just point out that this is the right way to solve a problem that we have in Ruby-based protocol implementations.<br>
Right now, there is no safe way on the Ruby level to use symbols to represent strings coming in on an unchecked interface.<br>
Of course, at the API level, I can easily use rb_check_id().<br>
The fact that this API exists should alert you to the fact that something is missing at the language level.</p>
<p>Kudos to Matthew for making this available as a gem for now.<br>
I'm not too wild about #interned as the name, but I'm used to Ruby method names being slightly idiosyncratic.</p>
<p>➔ This needs to go into Ruby core sooner than later.</p>
<p>The "alternatives" mentioned:<br>
<a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Let symbols be garbage collected (Closed)" href="https://bugs.ruby-lang.org/issues/7791">#7791</a> is a pipe dream (well, something like it could be made to work with an allocator region concept).<br>
<a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a> is serious damage (it could also be implemented on top of a global allocator region setting, which is still damage).</p>
<p>But why allocate at all if you <em>know</em> you don't want to?</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=421882013-10-02T10:15:21ZStudent (Nathan Zook)blogger@pierian-spring.net
<ul></ul><p>This was set to "next minor" a LONG time ago, but I don't see it in 2.1. ??? This would aid security in a couple of ways.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=424042013-10-10T11:53:17ZStudent (Nathan Zook)blogger@pierian-spring.net
<ul></ul><p>Is this feature request rejected? I thought it would be in 2.1</p>
<p>On 10/01/2013 06:15 PM, Student (Nathan Zook) wrote:</p>
<blockquote>
<p>Issue <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: New method Symbol[string] (Closed)" href="https://bugs.ruby-lang.org/issues/7854">#7854</a> has been updated by Student (Nathan Zook).</p>
<p>This was set to "next minor" a LONG time ago, but I don't see it in 2.1. ??? This would aid security in a couple of ways.</p>
<hr>
<p>Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: New method Symbol[string] (Closed)" href="https://bugs.ruby-lang.org/issues/7854">#7854</a>: New method Symbol[string]<br>
<a href="https://bugs.ruby-lang.org/issues/7854#change-42188" class="external">https://bugs.ruby-lang.org/issues/7854#change-42188</a></p>
<p>Author: Student (Nathan Zook)<br>
Status: Open<br>
Priority: Normal<br>
Assignee: matz (Yukihiro Matsumoto)<br>
Category: core<br>
Target version: next minor</p>
<p>I propose a new class method [] on Symbol. If a symbol s already exists such that s.to_s == string, then s is returned. If not, nil is returned.</p>
<p>The inspiration for this method is a question I was asked, and the answer I was given: "Why would you want to turn a tainted string into a symbol?" "I don't--I want to access an existing symbol with tainted data". Symbol[] accesses the symbol table like hash[] accesses the elements of a hash.</p>
<p>I believe that this completely addresses the problems behind tickets <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: Let symbols be garbage collected (Closed)" href="https://bugs.ruby-lang.org/issues/7791">#7791</a> and <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a>. I believe that it is a more intuitive solution than my proposal <a class="issue tracker-2 status-1 priority-4 priority-default" title="Feature: Symbol.defined? and/or to_existing_symbol (Open)" href="https://bugs.ruby-lang.org/issues/7795">#7795</a>, and I believe that this will also be useful for YAML.safe_load and similar initiatives.</p>
</blockquote> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454662014-02-25T15:03:05Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-6 priority-4 priority-default closed" href="/issues/7839">Feature #7839</a>: Symbol.freeze_symbols</i> added</li></ul> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454692014-02-25T15:18:23Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Rejected</i></li></ul><p>I like the basic idea but the name <code>Symbol[]</code> is not descriptive.<br>
As I replied to <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a>, the method should be a variation of #intern.</p>
<p>Matz.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454712014-02-25T15:26:24Zshugo (Shugo Maeda)
<ul></ul><p>Yukihiro Matsumoto wrote:</p>
<blockquote>
<p>I like the basic idea but the name <code>Symbol[]</code> is not descriptive.<br>
As I replied to <a class="issue tracker-2 status-6 priority-4 priority-default closed" title="Feature: Symbol.freeze_symbols (Rejected)" href="https://bugs.ruby-lang.org/issues/7839">#7839</a>, the method should be a variation of #intern.</p>
</blockquote>
<p>How about Symbol.find?</p>
<p>I guess String#intern came from Lisp, and Common Lisp has find-symbol, which returns a symbol only when it's found in a table.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454722014-02-25T15:48:51Znaruse (Yui NARUSE)naruse@airemix.jp
<ul></ul><pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git a/string.c b/string.c
index 4e30cb3..1e26a25 100644
</span><span class="gd">--- a/string.c
</span><span class="gi">+++ b/string.c
</span><span class="p">@@ -8231,6 +8231,27 @@</span> str_scrub_bang(int argc, VALUE *argv, VALUE str)
/*
* call-seq:
<span class="gi">+ * Symbol.find(str) -> symbol or nil
+ *
+ * Return the related symbol if the symbol already exists.
+ * Return nil if not.
+ */
+
+static VALUE
+sym_find(VALUE dummy, VALUE sym)
+{
+ ID id = rb_check_id(&sym);
+
+ if (id) {
+ return ID2SYM(id);
+ }
+ else {
+ return Qnil;
+ }
+}
+
+/*
+ * call-seq:
</span> * sym == obj -> true or false
*
* Equality---If <i>sym</i> and <i>obj</i> are exactly the same
<span class="p">@@ -8787,6 +8808,7 @@</span> Init_String(void)
rb_undef_alloc_func(rb_cSymbol);
rb_undef_method(CLASS_OF(rb_cSymbol), "new");
rb_define_singleton_method(rb_cSymbol, "all_symbols", rb_sym_all_symbols, 0); /* in parse.y */
<span class="gi">+ rb_define_singleton_method(rb_cSymbol, "find", sym_find, 1);
</span> rb_define_method(rb_cSymbol, "==", sym_equal, 1);
rb_define_method(rb_cSymbol, "===", sym_equal, 1);
<span class="gh">diff --git a/test/ruby/test_symbol.rb b/test/ruby/test_symbol.rb
index 7f261b6..cebaf43 100644
</span><span class="gd">--- a/test/ruby/test_symbol.rb
</span><span class="gi">+++ b/test/ruby/test_symbol.rb
</span><span class="p">@@ -1,4 +1,5 @@</span>
require 'test/unit'
<span class="gi">+require_relative 'envutil'
</span>
class TestSymbol < Test::Unit::TestCase
# <a href="https://blade.ruby-lang.org/ruby-core/3573">[ruby-core:3573]</a>
<span class="p">@@ -206,4 +207,12 @@</span> class TestSymbol < Test::Unit::TestCase
assert_equal(true, "foo#{Time.now.to_i}".to_sym.frozen?)
assert_equal(true, :foo.to_sym.frozen?)
end
<span class="gi">+
+ def test_sym_find
+ assert_separately(%w[--disable=gems], <<-"end;")
+ assert_equal :intern, Symbol.find("intern")
+ assert_equal nil, Symbol.find("hoge")
+ assert_raise(TypeError){ Symbol.find(true) }
+ end;
+ end
</span> end
</code></pre> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454742014-02-25T16:13:25Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul><li><strong>Status</strong> changed from <i>Rejected</i> to <i>Assigned</i></li><li><strong>Assignee</strong> changed from <i>matz (Yukihiro Matsumoto)</i> to <i>naruse (Yui NARUSE)</i></li></ul><p>Symbol.find is OK for me.</p>
<p>Matz.</p> Ruby master - Feature #7854: New method Symbol[string]https://bugs.ruby-lang.org/issues/7854?journal_id=454752014-02-25T16:34:12Znaruse (Yui NARUSE)naruse@airemix.jp
<ul><li><strong>Status</strong> changed from <i>Assigned</i> to <i>Closed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset r45175.</p>
<hr>
<ul>
<li>string.c (sym_find): Add Symbol.find(str), which returns whether given<br>
string is defined as symbol or not. [Feature <a class="issue tracker-2 status-5 priority-4 priority-default closed" title="Feature: New method Symbol[string] (Closed)" href="https://bugs.ruby-lang.org/issues/7854">#7854</a>]</li>
</ul>