Project

General

Profile

Actions

Bug #7698

closed

RubyGems 2.0 has an incompatibility about installation of extension libraries

Added by mrkn (Kenta Murata) almost 12 years ago. Updated over 11 years ago.

Status:
Closed
Target version:
ruby -v:
ruby 2.0.0dev (2013-01-14 trunk 38812) [x86_64-linux]
Backport:
[ruby-core:51437]

Description

The current rubygem included in ruby-head has an incompatibility about installation of extension libraries.
This incompatibility makes make install to be failed because the current gem doesn't separate build, source and install directories.

This is reported as a bug of bigdecimal's gemspec in #7344 firstly, but Tadashi Saito pointed out it also can be reproduced in other gems including extension libraries, such as decimal.gem.
I think this incompatibility will make gem-developers to be in trouble after ruby-2.0 is released.
So this should be resolved before ruby-2.0 is released.


Files


Related issues 5 (0 open5 closed)

Related to Ruby master - Bug #7344: gem pristine bigdecimal が失敗してしまうClosedmrkn (Kenta Murata)11/13/2012Actions
Related to Ruby master - Bug #6904: make -j all fails (sometimes)Closednobu (Nobuyoshi Nakada)08/22/2012Actions
Related to Backport200 - Backport #7991: backport r39542Closednagachika (Tomoyuki Chikanaga)03/01/2013Actions
Has duplicate Ruby master - Bug #7971: bson_ext doesn't install correctly in Ruby 2.0.0-p0Closednobu (Nobuyoshi Nakada)02/26/2013Actions
Has duplicate Ruby master - Bug #7949: Gem::Ext::Builder removes dependency to directory creation taskCloseddrbrain (Eric Hodel)02/25/2013Actions

Updated by mrkn (Kenta Murata) almost 12 years ago

Sorry, I made misspelling.

This incompatibility makes make install to be failed because the current gem doesn't separate build, source and install directories.

This incompatibility makes make install to be failed because the current rubygem doesn't separate build, source and install directories.

Updated by sorah (Sorah Fukumori) almost 12 years ago

Translating from [ruby-dev:46864], Tadashi said:

We can avoid this bug by changing directory tree in gems.
But, Current status means any gems may be not installable after 2.0.0, by incompatibility in current rubygem bundled in 2.0.0.

So, I propose the following solutions:

  1. Revert that incompatibility before 2.0.0 release or 2.0.0's first patch level release to make such gems installable.
  2. If you cannot, please review all gems that installable in 2.0.0 and notify to authors if a gem can't be installed in 2.0.0.
    (This may be rubygems.org's task)
  3. Announce that like "Your gem may be not installable in 2.0.0" to all gem authors.

I think solution (1) is the best. (2) seems high-cost.
(3) is the worst way for me, because I've not seen an announcement like such,
and seems too late to announce.

Updated by mrkn (Kenta Murata) almost 12 years ago

Thank you translation, sorah!

Updated by kou (Kouhei Sutou) almost 12 years ago

This can be fixed by the following patch. This is caused by #6904 change. Should this problem be fixed in RubyGems?

Index: lib/rubygems/ext/builder.rb

--- lib/rubygems/ext/builder.rb (revision 38833)
+++ lib/rubygems/ext/builder.rb (working copy)
@@ -19,6 +19,7 @@
mf = Gem.read_binary 'Makefile'
mf = mf.gsub(/^RUBYARCHDIR\s*=\s*$[^$]/, "RUBYARCHDIR = #{dest_path}")
mf = mf.gsub(/^RUBYLIBDIR\s
=\s*$[^$]*/, "RUBYLIBDIR = #{dest_path}")

  • mf = mf.gsub(/\s*\S+.time$/, "")

    File.open('Makefile', 'wb') {|f| f.print mf}

Updated by drbrain (Eric Hodel) almost 12 years ago

I am unfamiliar with Makefiles, so what does this patch change?

Also, .time is new in ruby 2.0, what effect does this have on ruby 1.9 and ruby 1.8?

I trust your judgement in adding this to RubyGems. If you commit it to Ruby please commit it to RubyGems and let me know!

Actions #6

Updated by kou (Kouhei Sutou) almost 12 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r38864.
Kenta, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/rubygems/ext/builder.rb (Gem::Ext::Builder.make): Remove
    .time dependency from *.rb install target. It causes needless
    *.rb install. [Bug #7698] [ruby-core:51437]
    Reported by Tadashi Saito. Thanks!!!
  • test/rubygems/test_gem_installer.rb
    (TestGemInstaller#test_install_extension_and_script): Add a test
    for the above change.

Updated by kou (Kouhei Sutou) almost 12 years ago

drbrain (Eric Hodel) wrote:

Also, .time is new in ruby 2.0, what effect does this have on ruby 1.9 and ruby 1.8?

This change doesn't have effect on 1.9 and 1.8 because Makefile on 1.9 and 1.8 doesn't have .time. So the gsub is just ignored.

I trust your judgement in adding this to RubyGems. If you commit it to Ruby please commit it to RubyGems and let me know!

Thanks. :-)
I've committed this change to Ruby and RubyGems with a test case.
Detail is written in commit message of the commit at the RubyGems repository: https://github.com/rubygems/rubygems/commit/a7cd1be541f2194fc8c1436d86bc05eb3a94500d

Updated by mame (Yusuke Endoh) over 11 years ago

  • Status changed from Closed to Assigned
  • Assignee changed from kou (Kouhei Sutou) to drbrain (Eric Hodel)

Hello,

This issue seems not to be fixed completely.
"gem install rbtree" still hit this issue.
It reproduces on both trunk and 2.0.0-p0.

$ gem install rbtree
Building native extensions. This could take a while...
ERROR: Error installing rbtree:
ERROR: Failed to build gem native extension.

/home/mame/local/bin/ruby extconf.rb

checking for ruby/st.h... yes
checking for rb_exec_recursive() in ruby.h... yes
creating Makefile

make
compiling rbtree.c
compiling dict.c
linking shared-object rbtree.so

make install
/usr/bin/install -c -m 0755 rbtree.so /home/mame/local/lib/ruby/gems/2.0.0/gems/rbtree-0.3.0/.
/usr/bin/install: rbtree.so' and /home/mame/local/lib/ruby/gems/2.0.0/gems/rbtree-0.3.0/./rbtree.so' are the same file
make: *** [install-so] Error 1

Gem files will remain installed in /home/mame/local/lib/ruby/gems/2.0.0/gems/rbtree-0.3.0 for inspection.
Results logged to /home/mame/local/lib/ruby/gems/2.0.0/gems/rbtree-0.3.0/./gem_make.out

--
Yusuke Endoh

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • File 0001-ext_conf_builder.rb-build-in-separate-dir.patch added

It's not good idea to mess up the source directory.

Actions #11

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • File deleted (0001-ext_conf_builder.rb-build-in-separate-dir.patch)

Updated by drbrain (Eric Hodel) over 11 years ago

Committed r39542. Thank you nobu for the RubyGems patch!

Updated by drbrain (Eric Hodel) over 11 years ago

  • Assignee changed from drbrain (Eric Hodel) to nobu (Nobuyoshi Nakada)
  • % Done changed from 100 to 50

=begin
This patch causes the rdiscount extension to fail to install since it reads a local file:

https://github.com/rtomayko/rdiscount/blob/master/ext/extconf.rb#L18

The exception:

Building native extensions. This could take a while...
/usr/local/bin/ruby -r./siteconf /Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1/ext/extconf.rb
checking for random()... yes
checking for srandom()... yes
checking for rand()... yes
checking for srand()... yes
checking size of unsigned long... 8
checking size of unsigned int... 4
checking size of unsigned int... 4
*** /Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1/ext/extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=/Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1/ext
--curdir
--ruby=/usr/local/bin/ruby
--with-rdiscount-dir
--without-rdiscount-dir
--with-rdiscount-include
--without-rdiscount-include=${rdiscount-dir}/include
--with-rdiscount-lib
--without-rdiscount-lib=${rdiscount-dir}/
/Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1/ext/extconf.rb:18:in read': No such file or directory - VERSION (Errno::ENOENT) from /Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1/ext/extconf.rb:18:in '
ERROR: Error installing rdiscount:
ERROR: Failed to build gem native extension.

  Building has failed. See above output for more information on the failure.

Gem files will remain installed in /Users/drbrain/tmp/gems/gems/rdiscount-2.0.7.1 for inspection.

=end

Updated by naruse (Yui NARUSE) over 11 years ago

It also breaks a test.
http://u64.rubyci.org/~chkbuild/ruby-trunk/log/20130301T010202Z.log.html.gz

  1. Error:
    test_install_extension_and_script(TestGemInstaller):
    Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.
/home/chkbuild/build/20130301T010202Z/ruby/ruby -r./siteconf /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/extconf.rb

creating Makefile

make
make[1]: Entering directory /home/chkbuild/build/20130301T010202Z/tmp/gem-install.20130301-11578-1y4vzyn' make[1]: Nothing to be done for all'.
make[1]: Leaving directory `/home/chkbuild/build/20130301T010202Z/tmp/gem-install.20130301-11578-1y4vzyn'

make install
make[1]: Entering directory /home/chkbuild/build/20130301T010202Z/tmp/gem-install.20130301-11578-1y4vzyn' installing default libraries make[1]: Circular /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/lib/a.rb <- /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/lib/a.rb dependency dropped. /usr/bin/install: /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/lib/a.rb' and /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/lib/a.rb' are the same file make[1]: *** [/home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/lib/a.rb] Error 1 make[1]: Leaving directory /home/chkbuild/build/20130301T010202Z/tmp/gem-install.20130301-11578-1y4vzyn'

Gem files will remain installed in /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2 for inspection.
Results logged to /home/chkbuild/build/20130301T010202Z/tmp/test_rubygems_11578/gemhome/gems/a-2/./gem_make.out

/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/builder.rb:55:in `run'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/builder.rb:28:in `block in make'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/builder.rb:26:in `each'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/builder.rb:26:in `make'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/ext_conf_builder.rb:33:in `block (2 levels) in build'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/ext_conf_builder.rb:20:in `chdir'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/ext_conf_builder.rb:20:in `block in build'
/home/chkbuild/build/20130301T010202Z/ruby/lib/tmpdir.rb:88:in `mktmpdir'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/ext/ext_conf_builder.rb:19:in `build'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:678:in `block (2 levels) in build_extensions'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:677:in `chdir'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:677:in `block in build_extensions'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:652:in `each'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:652:in `build_extensions'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/installer.rb:218:in `install'
/home/chkbuild/build/20130301T010202Z/ruby/test/rubygems/test_gem_installer.rb:1046:in `block in test_install_extension_and_script'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/user_interaction.rb:40:in `use_ui'
/home/chkbuild/build/20130301T010202Z/ruby/lib/rubygems/user_interaction.rb:63:in `use_ui'
/home/chkbuild/build/20130301T010202Z/ruby/test/rubygems/test_gem_installer.rb:1042:in `test_install_extension_and_script'

Updated by drbrain (Eric Hodel) over 11 years ago

What make do you have?

I have GNU Make 3.81 and cannot reproduce.

Actions #17

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r39558.
Kenta, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


ext_conf_builder.rb: build in the source directory

  • lib/rubygems/ext/ext_conf_builder.rb (Gem::Ext::ExtConfBuilder.build):
    revert use of temporary directory for build, to work some buggy
    extconf.rb which cannot build outside the source directory.
    [ruby-core:53056] [Bug #7698]

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Closed to Assigned
  • Target version changed from 2.0.0 to 2.1.0
  • % Done changed from 100 to 50
Actions #19

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r39559.
Kenta, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


ext_conf_builder.rb: use RUBYOPT

  • lib/rubygems/ext/ext_conf_builder.rb (Gem::Ext::ExtConfBuilder.build):
    use RUBYOPT instead of -r option, and revert some tests. [Bug #7698]

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Closed to Assigned
  • % Done changed from 100 to 50
Actions #21

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r39560.
Kenta, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


ext_conf_builder.rb: hack for obsolete sytle gems

  • lib/rubygems/ext/ext_conf_builder.rb
    (Gem::Ext::ExtConfBuilder.hack_for_obsolete_sytle_gems): remove
    circular dependencies for old style gems which locate extconf.rb on
    the toplevel. [ruby-core:53059] [ruby-trunk - Bug #7698]

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Closed to Assigned
  • % Done changed from 100 to 50
Actions #23

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 50 to 100

This issue was solved with changeset r39572.
Kenta, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


ext_conf_builder.rb: remove circular dependency in install-so

  • lib/rubygems/ext/ext_conf_builder.rb
    (Gem::Ext::ExtConfBuilder.hack_for_obsolete_style_gems): remove
    circular dependencies in install-so too. [ruby-core:52882]
    [Bug #7698]

Updated by zzak (zzak _) over 11 years ago

I can still reproduce this on RubyGems 2.0.3 with trunk:
ruby 2.1.0dev (2013-06-16 trunk 41328) [x86_64-darwin12.3.0]

To repro: gem install rbtree

Is this a bug in rbtree?

Updated by nagachika (Tomoyuki Chikanaga) over 11 years ago

Hello,

The following is the output of gem install rbtree on my environment (ruby 2.1.0dev (2013-06-16 trunk 41337) [x86_64-darwin10.8.0])
It seems that rbtree need to catch up with constify of RBasic::klass and RHash::ifnone for RGenGC.
ko1, what do you think? Is it intended incompatibility?

% gem install rbtree
Fetching: rbtree-0.4.1.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing rbtree:
ERROR: Failed to build gem native extension.

/Users/nagachika/opt/ruby-trunk/bin/ruby-trunk extconf.rb

checking for ruby/st.h... yes
checking for rb_exec_recursive() in ruby.h... yes
checking for rb_exec_recursive_paired() in ruby.h... yes
checking for rb_proc_lambda_p() in ruby.h... yes
creating Makefile

make
compiling dict.c
compiling rbtree.c
rbtree.c: In function ‘copy_dict’:
rbtree.c:755: error: assignment of read-only member ‘klass’
rbtree.c: In function ‘rbtree_to_hash’:
rbtree.c:1292: error: assignment of read-only member ‘ifnone’
make: *** [rbtree.o] Error 1

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

It's a completely different issue, and intended incompatibility AFAIK.

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0