Project

General

Profile

Bug #16680

Symlink folder in $LOAD_PATH does not work with autoload

Added by zw963 (Wei Zheng) 9 months ago. Updated 3 months ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:97410]

Description

The following is a full reproducible process. I will use a gem named 'looksee' to reproduce this.

  1. Do gem install looksee

  2. Go to gem folder and copy lib content into /tmp/test_looksee so that we have a folder like this:

     ╰─ $ tree /tmp/test_looksee
    /tmp/test_looksee
    └── lib
        ├── looksee
        │   ├── adapter
        │   │   ├── base.rb
        │   │   └── rubinius.rb
        │   ├── adapter.rb
        │   ├── clean.rb
        │   ├── columnizer.rb
        │   ├── core_ext.rb
        │   ├── editor.rb
        │   ├── help.rb
        │   ├── inspector.rb
        │   ├── lookup_path.rb
        │   ├── mri.so
        │   └── version.rb
        └── looksee.rb
    
  3. Create a new symlink to this folder.

     ╰─ $ln -s /tmp/test_looksee /tmp/test_looksee1
    
     ╰─ $ ls -alhd /tmp/test_looksee*
    drwxr-xr-x 3 zw963 zw963 60 2020-03-08 01:39 /tmp/test_looksee/
    lrwxrwxrwx 1 zw963 zw963 13 2020-03-08 02:08 /tmp/test_looksee1 -> 
    test_looksee//
    
  4. Run the following command to reproduce this issue.

     ╰─ $ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
    Traceback (most recent call last):
            10: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         9: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         8: from /tmp/test_looksee1/lib/looksee.rb:1:in `<top (required)>'
         7: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         6: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         5: from /tmp/test_looksee1/lib/looksee/clean.rb:4:in `<top (required)>'
         4: from /tmp/test_looksee1/lib/looksee/clean.rb:171:in `<module:Looksee>'
         3: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         2: from /home/zw963/others/.rvm/rubies/ruby-2.7.0/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
         1: from /tmp/test_looksee1/lib/looksee/adapter.rb:1:in `<top (required)>'
    /tmp/test_looksee1/lib/looksee/adapter.rb:2:in `<module:Looksee>': uninitialized constant Looksee::Adapter (NameError)
    

You may want to ask why use symlinks folder in $LOAD_PATH. I think this is a quite common case; many ruby developers have a local gem not managed by Rubygems or Bundler. They just use $LOAD_PATH to require/load it, and it is hard to ensure everyone's folder is not a symlink.

I think this is a breaking change. All those codes worked quite well before 2.7 because old codes always
follow symlinks. The following is an example from ruby 2.6.3, which works.

 ╰─ $ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
Traceback (most recent call last):
        8: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        7: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        6: from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
        5: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        4: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
        3: from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
        2: from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
        1: from /home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/home/zw963/others/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': incompatible library version - /tmp/test_looksee/lib/looksee/mri.so (LoadError)

As we can seen, those breaking codes come from autoload :Adapter, 'looksee/adapter'. I suspect
maybe this is caused by nested autoload. But this is only a guess, please check. Thank you.


Files

clipboard-202003080224-xglcq.png (183 KB) clipboard-202003080224-xglcq.png zw963 (Wei Zheng), 03/07/2020 06:23 PM

Related issues

Related to Ruby master - Bug #16462: Ruby 2.7 autoload not working with $RUBYLIB (maybe circular dependency error)ClosedActions
#1

Updated by hsbt (Hiroshi SHIBATA) 9 months ago

  • Related to Bug #16462: Ruby 2.7 autoload not working with $RUBYLIB (maybe circular dependency error) added

Updated by zw963 (Wei Zheng) 7 months ago

Sorry, any update on this?

Updated by Eregon (Benoit Daloze) 7 months ago

Maybe a solution is to always resolve symlinks in the cache of $LOAD_PATH?
One issue is that $LOAD_PATH entries might not necessarily exist.

Updated by zw963 (Wei Zheng) 6 months ago

Eregon (Benoit Daloze) wrote in #note-3:

One issue is that $LOAD_PATH entries might not necessarily exist.

Sorry, not really understood what your means about reference comment.

I think maybe it acceptable for only support non-symlink $LOAD_PATH when update to ruby 2.7.
but, it should be mentions on release change log and mention why we change this behavior
before ruby 2.7 release.

Updated by nobu (Nobuyoshi Nakada) 6 months ago

I can't reproduce it with the master.

$ ruby -v
ruby 2.8.0dev (2020-06-13T14:33:40Z master 2496bdb28f) [x86_64-linux]

First, copied looksee/lib directory to /tmp/test_looksee, and linked it to /tmp/test_looksee1, correct?

$ ls -lRA /tmp/test_looksee*
lrwxrwxrwx 1 ubuntu ubuntu   17 Jun 14 07:49 /tmp/test_looksee1 -> /tmp/test_looksee/

/tmp/test_looksee:
total 4
drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 07:55 lib/

/tmp/test_looksee/lib:
total 8
drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 07:42 looksee/
-rw-r--r-- 1 ubuntu ubuntu   51 Jun 14 07:42 looksee.rb

/tmp/test_looksee/lib/looksee:
total 44
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 14 07:42 adapter/
-rw-r--r-- 1 ubuntu ubuntu  246 Jun 14 07:42 adapter.rb
-rw-r--r-- 1 ubuntu ubuntu 5276 Jun 14 07:42 clean.rb
-rw-r--r-- 1 ubuntu ubuntu 2107 Jun 14 07:42 columnizer.rb
-rw-r--r-- 1 ubuntu ubuntu  732 Jun 14 07:42 core_ext.rb
-rw-r--r-- 1 ubuntu ubuntu 1522 Jun 14 07:42 editor.rb
-rw-r--r-- 1 ubuntu ubuntu 1875 Jun 14 07:42 help.rb
-rw-r--r-- 1 ubuntu ubuntu 2046 Jun 14 07:42 inspector.rb
-rw-r--r-- 1 ubuntu ubuntu 2292 Jun 14 07:42 lookup_path.rb
-rw-r--r-- 1 ubuntu ubuntu  128 Jun 14 07:42 version.rb

/tmp/test_looksee/lib/looksee/adapter:
total 8
-rw-r--r-- 1 ubuntu ubuntu 2018 Jun 14 07:42 base.rb
-rw-r--r-- 1 ubuntu ubuntu  629 Jun 14 07:42 rubinius.rb

Second, run with the real path, and got a LoadError on mri.so as expected.

$ ruby -I/tmp/test_looksee/lib/ -rlooksee -e 'puts 100'
/home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- looksee/mri.so (LoadError)
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
    from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
bash: exit 1

Then, tried the symlink path, got the same error.

$ ruby -I/tmp/test_looksee1/lib/ -rlooksee -e 'puts 100'
/home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require': cannot load such file -- looksee/mri.so (LoadError)
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /tmp/test_looksee/lib/looksee/clean.rb:171:in `<module:Looksee>'
    from /tmp/test_looksee/lib/looksee/clean.rb:4:in `<top (required)>'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /tmp/test_looksee/lib/looksee.rb:1:in `<top (required)>'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
    from /home/ubuntu/.rbenv/versions/master-2020-06-13/lib/ruby/2.8.0/rubygems/core_ext/kernel_require.rb:83:in `require'
bash: exit 1

Updated by nobu (Nobuyoshi Nakada) 6 months ago

I could reproduce it with 2.7.1 and 2.7 head.

Updated by zw963 (Wei Zheng) 3 months ago

nobu (Nobuyoshi Nakada) wrote in #note-6:

I can't reproduce it with the master.

I could reproduce it with 2.7.1 and 2.7 head.

Cool, good news, so, it was fixed on our master, right?

#8

Updated by sawa (Tsuyoshi Sawada) 3 months ago

  • Description updated (diff)
  • Subject changed from [Breaking Change] Ruby 2.7 not support symlinks folder in $LOAD_PATH to work with autoload. to Symlink folder in $LOAD_PATH does not work with autoload

Also available in: Atom PDF