Backport #4235: svn keywords in code prevent correct building of ruby using git mirror

• Status: Closed
• Assignee: shyouhei (Shyouhei Urabe)

[stepheneb (Stephen Bannasch)]

=begin
Right now there are 355 locations in C and Ruby source files where Subversion keywords are used.

$ ack -l '$(Date|Revision|Author|HeadURL|Id)$' --nohtml | wc -l
355

Almost all of these uses are in comments in source files.

However there are a number of places that $Id$ and $Revision$ are used in runnable code.

This means that correct Ruby builds can't be made using the official git mirror: https://github.com/ruby/ruby

The usage breaks down as follows:

$ ack -l '$(Date)$' --nohtml | wc -l
1
$ ack -l '$(Revision)$' --nohtml | wc -l
58
$ ack -l '$(Author)$' --nohtml | wc -l
113
$ ack -l '$(HeadURL)$' --nohtml | wc -l
0
$ ack -l '$(Id)$' --nohtml | wc -l
212

The most useful first step would be to remove the use of subversion keywords from the runnable code.

Here are the locations I found usage of svn keywords in runnable code in trunk. This is done in teh trunk branch from a git clone of the official mirror: https://github.com/ruby/ruby

I used the following regexes with ack to at least exclude usage in ruby commenetd code that starts with '#':

ack '^[^#].$Id$' --nohtml --heading --break -C 6
ack '^[^#].$Revision$' --nohtml --heading --break -C 6

http.rb, pop.rb, and smtp.rb:

class HTTP < Protocol

 # :stopdoc:
 Revision = %q$Revision$.split[1]
 HTTPVersion = '1.1'

rexml.rb:

REVISION = %w$Revision$[1] || ''

ruby-mode-el and ruby-style.el:

(defconst ruby-mode-revision "$Revision$"
"Ruby mode revision string.")

in lib/irb: cmd/fork.rb, cmd/nop.rb, completion.rb, ext/history.rb, ext/loader.rb,
ext/multi-irb.rb, ext/save-history.rb, input-method.rb, locale.rb, output-method.rb,
ruby-lex.rb, slex.rb, xmp.rb and irb.rb itself:

@RCS_ID='-$Id$-'

lib/logger.rb

_, name, rev = %w$Id$

lib/optparse.rb

RCSID = %w$Id$[1..-1].each {|s| s.freeze}.freeze

lib/sync.rb

RCS_ID='-$Id$-'

lib/net/telnet.rb

REVISION = '$Id$'

lib/cgi/core.rb

REVISION = '$Id$' #:nodoc:

ext/strscan/strscan.c

tmp = rb_str_new2("$Id$");

There are some places where $Id$ is used in code buit the code itself is commented out:

ext/sdbm/_sdbm.c

#ifndef lint
/char sdbm_rcsid[] = "$Id$";/
#endif

and:

#ifndef lint
/char pair_rcsid[] = "$Id$";/
#endif

---

Here's a summary of some of the ways svn keywords are used now in comments in code:

Ruby source:

fork.rb -

$Release Version: 0.9.6 $

$Revision$

by Keiju ISHITSUKA([e-mail])

or a C header:

/**********************************************************************

 rubysig.h -

 $Author$
 $Date$
 created at: Wed Aug 16 01:15:38 JST 1995

 Copyright (C) 1993-2008 Yukihiro Matsumoto

**********************************************************************/

Or an emacs script (misc/inf-ruby.el):

;;; --Emacs-Lisp--
;;;
;;; $Id$
;;; $Author$

Or block comments using =begin (lib/xmlrpc/datetime.rb):

=begin
= History
$Id$
=end
=end

[meta (mathew murphy)]

=begin
Also makes Ruby tests fail when building using a bzr pull from the SVN repo.

i.e. this issue basically means that only SVN can be used to obtain a fully working copy of the source tree that passes the tests.
=end

[Cezary (Cezary Baginski)]

=begin
Newest version of RVM (1.1.13) installs from github:

$ rvm get head; rvm install ruby-head

...

Result:

~/.rvm/src/ruby-head $ ruby --version
ruby 1.9.3dev (2011-01-06 trunk 30471) [x86_64-linux]
~/.rvm/src/ruby-head $ git branch -v

• trunk 5631792 * ext/-test-/array/resize/resize.c (Init_resize): renamed method for test.

Test results:

~/.rvm/src/ruby-head $ make test-all

...

8632 tests, 2207190 assertions, 6 failures, 1 errors, 0 skips
make: *** [yes-test-all] Błąd 7

Possible reasons why it works:

• works on head
• RVM does some magic
• both
  =end

[stepheneb (Stephen Bannasch)]

=begin
I updated rvm and used it to install a trunk version of Ruby which it cloned with git:

$ rvm use ruby-head
Using /Users/stephen/.rvm/gems/ruby-head
running hook after_use

$ ruby -v
ruby 1.9.3dev (2011-01-06 trunk 30471) [x86_64-darwin10.5.0]

The original form of the subversion keywords are still located in the source files.

For example line 211 of lib/optparse.rb:

$ grep -n -C 3 '$Id$' /Users/stephen/.rvm/rubies/ruby-head/lib/ruby/1.9.1/optparse.rb
208-#
209-class OptionParser
210- # :stopdoc:
211: RCSID = %w$Id$[1..-1].each {|s| s.freeze}.freeze
212- Version = (RCSID[1].split('.').collect {|s| s.to_i}.extend(Comparable).freeze if RCSID[1])
213- LastModified = (Time.gm(*RCSID[2, 2].join('-').scan(/\d+/).collect {|s| s.to_i}) if RCSID[2])
214- Release = RCSID[2]

But it turns out that the constants: Version LastModified are just set to nil -- no errors are thrown. In the past missing keyword substitution caused runtime errors. It appears that is no longer the case.

usage in optparse.rb:

RCSID = %w$Id$[1..-1].each {|s| s.freeze}.freeze
=> []
Version = (RCSID[1].split('.').collect {|s| s.to_i}.extend(Comparable).freeze if RCSID[1])
=> nil
LastModified = (Time.gm(*RCSID[2, 2].join('-').scan(/\d+/).collect {|s| s.to_i}) if RCSID[2])
=> nil

usage in http.rb, pop.rb, and smtp.rb:

Revision = %q$Revision$.split[1]
=> nil

usage in rexml.rb:

REVISION = %w$Revision$[1] || ''
=> ""

usage in lib/logger.rb:

_, name, rev = %w$Id$
=> ["Id"]

=end

[zimbatm (zimba tm)]

=begin
It's possible to expand some keywords with git, but it's an advanced feature. Maybe this might help, even if in the long time, it's probably better to rely less on these:

http://progit.org/book/ch7-2.html#keyword_expansion
=end

[naruse (Yui NARUSE)]

=begin
2011/1/6 Stephen Bannasch [e-mail]:

Right now there are 355 locations in C and Ruby source files where Subversion keywords are used.

We know many CVS/Subversion keywords are remained.
But all keywords which raises error were already fixed.

--
NARUSE, Yui
[e-mail]

=end

[naruse (Yui NARUSE)]

=begin
2011/1/6 mathew murphy [e-mail]:

Issue #4235 has been updated by mathew murphy.

Also makes Ruby tests fail when building using a bzr pull from the SVN repo.

i.e. this issue basically means that only SVN can be used to obtain a fully working copy of the source tree that passes the tests.

What's the test?

--
NARUSE, Yui
[e-mail]

=end

[stepheneb (Stephen Bannasch)]

=begin
I ran: 'make test' in the src directory created when rvm used git clone on the trunk branch and everything passed.

I still think removing the svn keywords would be good but I would change the priority of this ticket to low.

[ruby-head (trunk)]$ pwd
/Users/stephen/.rvm/src/ruby-head

[ruby-head (trunk)]$ ls -l configure
-rwxr-xr-x 1 stephen staff 1062077 Jan 5 18:52 configure

[ruby-head (trunk)]$ make test

sample/test.rb:assignment

...

 test_thread.rb ..................................................

PASS all 934 tests
./miniruby -I./lib -I.ext/common ./tool/runruby.rb --extout=.ext -- "./bootstraptest/runner.rb" --ruby="ruby" ./KNOWNBUGS.rb
2011-01-05 22:33:50 -0500
Driver is ruby 1.9.3dev (2011-01-06 trunk 30471) [x86_64-darwin10.5.0]
Target is ruby 1.9.3dev (2011-01-06 trunk 30471) [x86_64-darwin10.5.0]

KNOWNBUGS.rb .
PASS all 1 tests

=end

[naruse (Yui NARUSE)]

• Priority changed from Normal to 3

=begin
Please let us know if real problem still remain.
=end

[meta (mathew murphy)]

=begin
Still an issue for 1.8.7, perhaps this ticket should be moved there?
=end

[headius (Charles Nutter)]

=begin
Several of the changes on JRuby's "fork" of stdlib are specifically to fix this problem. For example:

diff --git a/lib/logger.rb b/lib/logger.rb
index 15d95fc..d5f5425 100644
--- a/lib/logger.rb
+++ b/lib/logger.rb
@@ -170,8 +170,8 @@ require 'monitor'

class Logger
VERSION = "1.2.6"

• id, name, rev = %w$Id$
• ProgName = "#{name.chomp(",v")}/#{rev}"

• this is faked to avoid the svn ID changing with every update

• ProgName = "logger.rb/99999"

  class Error < RuntimeError; end
  class ShiftingError < Error; end

=end

[naruse (Yui NARUSE)]

• Status changed from Open to Assigned
• Assignee set to shyouhei (Shyouhei Urabe)

=begin
Please comment to mathew's question.

Still an issue for 1.8.7, perhaps this ticket should be moved there?
=end

[shyouhei (Shyouhei Urabe)]

• Status changed from Assigned to Closed