Project

General

Profile

Bug #9106

'gem install' doesn't copy .so files of ext libs

Added by tagomoris (Satoshi TAGOMORI) almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Target version:
ruby -v:
ruby 2.1.0dev (2013-11-12 trunk 43653) [x86_64-linux]
[ruby-core:58306]

Description

'gem install' should copy files of 'ext/*/.so' of each installed gems, but doesn't.

$ ruby -v
ruby 2.1.0dev (2013-11-12 trunk 43653) [x86_64-linux]
$ gem install msgpack
Fetching: msgpack-0.5.7.gem (100%)
Building native extensions. This could take a while...
Successfully installed msgpack-0.5.7

[]

Parsing documentation for msgpack-0.5.7
Installing ri documentation for msgpack-0.5.7
Done installing documentation for msgpack after 0 seconds
1 gem installed
$ ruby -rmsgpack -e ''
/home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in require': cannot load such file -- /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack/msgpack (LoadError)
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in
require'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack.rb:8:in rescue in <top (required)>'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack.rb:3:in
'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in require'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:135:in
rescue in require'
from /home/user/local/ruby-2.1.0-trunk/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:144:in `require'
$ ls -l local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/lib/msgpack
total 4
-rw-r--r-- 1 edge-dev edge 42 Nov 13 15:43 version.rb
$ ls -l local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/ext/msgpack/msgpack.so
-rwxr-xr-x 1 edge-dev edge 1390955 Nov 13 15:43 local/ruby-2.1.0-trunk/lib/ruby/gems/2.1.0/gems/msgpack-0.5.7/ext/msgpack/msgpack.so

Associated revisions

Revision 43714
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:

Compatibly renamed Gem::DependencyResolver to Gem::Resolver.

Added support for git gems in gem.deps.rb and Gemfile.

Fixed resolver bugs.

  • test/rubygems: ditto.

  • lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
    [ruby-trunk - Bug #9086]

  • lib/rubygems/commands/which_command.rb: RubyGems now indicates
    failure when any file is missing. [ruby-trunk - Bug #9004]

  • lib/rubygems/ext/builder: Extensions are now installed into the
    extension install directory and the first directory in the require
    path from the gem. This allows backwards compatibility with msgpack
    and other gems that calculate full require paths.
    [ruby-trunk - Bug #9106]

Revision 43714
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:

Compatibly renamed Gem::DependencyResolver to Gem::Resolver.

Added support for git gems in gem.deps.rb and Gemfile.

Fixed resolver bugs.

  • test/rubygems: ditto.

  • lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
    [ruby-trunk - Bug #9086]

  • lib/rubygems/commands/which_command.rb: RubyGems now indicates
    failure when any file is missing. [ruby-trunk - Bug #9004]

  • lib/rubygems/ext/builder: Extensions are now installed into the
    extension install directory and the first directory in the require
    path from the gem. This allows backwards compatibility with msgpack
    and other gems that calculate full require paths.
    [ruby-trunk - Bug #9106]

Revision 43714
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:

Compatibly renamed Gem::DependencyResolver to Gem::Resolver.

Added support for git gems in gem.deps.rb and Gemfile.

Fixed resolver bugs.

  • test/rubygems: ditto.

  • lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
    [ruby-trunk - Bug #9086]

  • lib/rubygems/commands/which_command.rb: RubyGems now indicates
    failure when any file is missing. [ruby-trunk - Bug #9004]

  • lib/rubygems/ext/builder: Extensions are now installed into the
    extension install directory and the first directory in the require
    path from the gem. This allows backwards compatibility with msgpack
    and other gems that calculate full require paths.
    [ruby-trunk - Bug #9106]

Revision 43714
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:

Compatibly renamed Gem::DependencyResolver to Gem::Resolver.

Added support for git gems in gem.deps.rb and Gemfile.

Fixed resolver bugs.

  • test/rubygems: ditto.

  • lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
    [ruby-trunk - Bug #9086]

  • lib/rubygems/commands/which_command.rb: RubyGems now indicates
    failure when any file is missing. [ruby-trunk - Bug #9004]

  • lib/rubygems/ext/builder: Extensions are now installed into the
    extension install directory and the first directory in the require
    path from the gem. This allows backwards compatibility with msgpack
    and other gems that calculate full require paths.
    [ruby-trunk - Bug #9106]

Revision 43953
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master baa965b. Notable changes:

Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106]

  • test/rubygems: ditto.

Revision 43953
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master baa965b. Notable changes:

Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106]

  • test/rubygems: ditto.

Revision 43953
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master baa965b. Notable changes:

Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106]

  • test/rubygems: ditto.

Revision 43953
Added by drbrain (Eric Hodel) almost 4 years ago

  • lib/rubygems: Update to RubyGems master baa965b. Notable changes:

Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106]

  • test/rubygems: ditto.

History

#1 [ruby-core:58307] Updated by sorah (Sorah Fukumori) almost 4 years ago

  • Status changed from Open to Assigned
  • Assignee set to drbrain (Eric Hodel)

Eric, could you handle this?

#2 [ruby-core:58317] Updated by drbrain (Eric Hodel) almost 4 years ago

  • Priority changed from 7 to Normal

Yes, I will investigate.

#3 [ruby-core:58318] Updated by drbrain (Eric Hodel) almost 4 years ago

=begin
Ugh. This is a misfeature of msgpack, it should trust that RubyGems will do the right thing:

here = File.expand_path(File.dirname(FILE))
require File.join(here, 'msgpack', 'version')
begin
m = /(\d+.\d+)/.match(RUBY_VERSION)
ver = m[1]
require File.join(here, 'msgpack', ver, 'msgpack')
rescue LoadError
require File.join(here, 'msgpack', 'msgpack')
end

=end

#4 [ruby-core:58347] Updated by hsbt (Hiroshi SHIBATA) almost 4 years ago

  • Status changed from Assigned to Feedback

tagomoris: Thank you for your report. Can you investigate to msgpack gem?

#5 [ruby-core:58353] Updated by drbrain (Eric Hodel) almost 4 years ago

  • Status changed from Feedback to Assigned
  • Backport changed from 1.9.3: UNKNOWN, 2.0.0: UNKNOWN to 1.9.3: DONTNEED, 2.0.0: DONTNEED

For backwards compatibility I will need to alter RubyGems before Ruby 2.1 (I haven't had time yet).

Future versions of RubyGems will not support this lack of trust, though.

#6 [ruby-core:58365] Updated by tagomoris (Satoshi TAGOMORI) almost 4 years ago

hsbt (Hiroshi SHIBATA) wrote:

tagomoris: Thank you for your report. Can you investigate to msgpack gem?

I don't understand why msgpack requires with such code. So I'll tell it to frsyuki (Sadayuki Furuhashi), original developer of msgpack.

#7 [ruby-core:58366] Updated by frsyuki (Sadayuki Furuhashi) almost 4 years ago

Hi I'm @frsyuki.
1) I don't understand which behavior of RubyGems should the code trust. It assumes that msgpack.so is placed into lib/msgpack/msgpack.so because ext/msgpack/extconf.rb includes this line:
create_makefile('msgpack/msgpack')

2) Which fix is appropriate?:
a) require File.join(here, '..', 'ext', 'msgpack', 'msgpack')
b) add "ext" directory to require_paths in gemspec
c) require "msgpack/msgpack"
d) require "msgpack#{RbConfig::CONFIG["DLEXT"]}"

#8 [ruby-core:58370] Updated by drbrain (Eric Hodel) almost 4 years ago

=begin
frsyuki (Sadayuki Furuhashi): you should trust rubygems to make msgpack/msgpack.so accessible somewhere in the $LOAD_PATH so:

require 'msgpack/msgpack'

so option c is the best fix.

With RubyGems 2.2 the directory you gem install to can be shared across multiple ruby versions because the extensions end up in $GEM_HOME/extensions///. RubyGems will add that directory to $LOAD_PATH so require can find files in it.

For msgpack, on OS X with ruby 2.1.0 this would be:

$GEM_HOME/extensions/x86_64-darwin-12/2.0.0/msgpack-0.5.7/msgpack/msgpack.bundle

I will figure out a backward compatible solution so older msgpack will still work for RubyGems 2.x. There is no reason for you to rush out a new release of msgpack.
=end

#9 [ruby-core:58373] Updated by frsyuki (Sadayuki Furuhashi) almost 4 years ago

It sounds good improvement of RubyGems to manage binaries for each ruby API versions.
Your comment totally makes sense. I'll fix msgpack next version. Thank you!

#10 [ruby-core:58379] Updated by luislavena (Luis Lavena) almost 4 years ago

Hello,

Here is a 2009 discussion about gems and multiple binaries (targeting different versions of Ruby):

http://rubyforge.org/pipermail/rubygems-developers/2009-April/004522.html

My alternative to that issue is what you can see in projects like sqlite3-ruby:

https://github.com/sparklemotion/sqlite3-ruby/blob/master/lib/sqlite3.rb#L1-L7

And others that follow the guidelines of rake-compiler to deal with these multiple binaries in a single gem.

Of course, this is to deal with gems that package binaries, gems that are compiled on installation, we let RubyGems do its job.

#11 Updated by drbrain (Eric Hodel) almost 4 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r43714.
Satoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/rubygems: Update to RubyGems master 6a3d9f9. Changes include:

Compatibly renamed Gem::DependencyResolver to Gem::Resolver.

Added support for git gems in gem.deps.rb and Gemfile.

Fixed resolver bugs.

  • test/rubygems: ditto.

  • lib/rubygems/LICENSE.txt: Updated to license from RubyGems trunk.
    [ruby-trunk - Bug #9086]

  • lib/rubygems/commands/which_command.rb: RubyGems now indicates
    failure when any file is missing. [ruby-trunk - Bug #9004]

  • lib/rubygems/ext/builder: Extensions are now installed into the
    extension install directory and the first directory in the require
    path from the gem. This allows backwards compatibility with msgpack
    and other gems that calculate full require paths.
    [ruby-trunk - Bug #9106]

#12 [ruby-core:58589] Updated by tagomoris (Satoshi TAGOMORI) almost 4 years ago

This problem is reproduced on ruby 2.1.0-preview2 on Linux/OSX.
'msgpack' gem cannot be loaded (LoadError) and 'ext/msgpack/msgpack.so' (or .bundle) is not copied to 'lib/msgpack/msgpack.so'.
About other library (ex: cool.io), binary files are copied to 'lib' directory.

P.S. I cannot reopen this ticket.

#13 [ruby-core:58591] Updated by nagachika (Tomoyuki Chikanaga) almost 4 years ago

  • Status changed from Closed to Assigned

#14 [ruby-core:58623] Updated by drbrain (Eric Hodel) almost 4 years ago

Hrm, I will check it.

#15 [ruby-core:58723] Updated by drbrain (Eric Hodel) almost 4 years ago

I found my bug!

I didn't recursively copy the files.

Fix soon!

#16 Updated by drbrain (Eric Hodel) almost 4 years ago

  • Status changed from Assigned to Closed

This issue was solved with changeset r43953.
Satoshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • lib/rubygems: Update to RubyGems master baa965b. Notable changes:

Copy directories to lib/ when installing extensions. This completes
the fix for [ruby-trunk - Bug #9106]

  • test/rubygems: ditto.

Also available in: Atom PDF