Project

General

Profile

Bug #10222

require_relative and require should be compatible with each other when symlinks are used

Added by rosenfeld (Rodrigo Rosenfeld Rosas) almost 4 years ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
2.3.1, 2.1.2p95
[ruby-core:64928]

Description

Not sure if this should be considered a bug or a feature request since I don't know whether the current behavior is intended or not.

Recently I got a report for my gem rails-web-console related to require_relative causing trouble with symlinked dirs:

https://github.com/rosenfeld/active_record_migrations/issues/6

Dmitry was able to replicate the issue using vanilla Ruby:

mkdir a
ln -s a b
echo "require_relative 'b'" > a/a.rb
echo "p 'b loaded'" > a/b.rb
echo "$: << File.expand_path('../b', __FILE__); require 'a'; require 'b'" > c.rb
ruby c.rb

Notice how "b loaded" is printed twice but if you replace require_relative with require it's just loaded once.

Shouldn't Ruby always expand the loaded files before appending them to the $LOADED_FEATURES and avoid this kind of error? I don't think require_relative should behave differently than a regular require in such cases.

Any thoughts?


Related issues

Related to Ruby trunk - Bug #14372: Memory leak in require with Pathnames in the $LOAD_PATH in 2.3/2.4Closed

Associated revisions

Revision b6d3927e
Added by nobu (Nobuyoshi Nakada) 10 months ago

load.c: real path to load

  • load.c (rb_construct_expanded_load_path): expand load paths to real paths to get rid of duplicate loading from symbolic-linked directories. [Feature #10222]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 59984
Added by nobu (Nobuyoshi Nakada) 10 months ago

load.c: real path to load

  • load.c (rb_construct_expanded_load_path): expand load paths to real paths to get rid of duplicate loading from symbolic-linked directories. [Feature #10222]

Revision 59984
Added by nobu (Nobuyoshi Nakada) 10 months ago

load.c: real path to load

  • load.c (rb_construct_expanded_load_path): expand load paths to real paths to get rid of duplicate loading from symbolic-linked directories. [Feature #10222]

Revision eaba9da1
Added by nagachika (Tomoyuki Chikanaga) 5 months ago

merge revision(s) 59983,59984: [Backport #10222] [Backport #14372] [Backport #14424]

file.c: rb_check_realpath

* file.c (rb_check_realpath): returns real path which has no
  symbolic links.  similar to rb_realpath except for returning
  Qnil if any parts did not exist.

load.c: real path to load

* load.c (rb_construct_expanded_load_path): expand load paths to
  real paths to get rid of duplicate loading from symbolic-linked
  directories.  [Feature #10222]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_4@62440 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 62440
Added by nagachika (Tomoyuki Chikanaga) 5 months ago

merge revision(s) 59983,59984: [Backport #10222] [Backport #14372] [Backport #14424]

file.c: rb_check_realpath

* file.c (rb_check_realpath): returns real path which has no
  symbolic links.  similar to rb_realpath except for returning
  Qnil if any parts did not exist.

load.c: real path to load

* load.c (rb_construct_expanded_load_path): expand load paths to
  real paths to get rid of duplicate loading from symbolic-linked
  directories.  [Feature #10222]

History

#1 [ruby-core:64932] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) almost 4 years ago

I can't change the title myself. Could someone with privileges please change it to something like: "require_relative and require should be compatible with each other when symlinks are used".

I think this would make it easier to be searchable if others are experiencing the same issue. The key change is to add the "symlinks" word to the title so that the connection is made clear.

#2 [ruby-core:65031] Updated by shevegen (Robert A. Heiler) almost 4 years ago

Curious behaviour indeed, there may be a reason why symlinks were assumed to behave differently.

#3 [ruby-core:76978] Updated by abolshakov (Tema Bolshakov) almost 2 years ago

  • ruby -v changed from 2.1.2p95 to 2.3.1

#4 [ruby-core:76979] Updated by abolshakov (Tema Bolshakov) almost 2 years ago

  • ruby -v changed from 2.3.1 to 2.3.1, 2.1.2p95

#5 [ruby-core:77573] Updated by shyouhei (Shyouhei Urabe) almost 2 years ago

  • Subject changed from require_relative and require should be compatible with each other to require_relative and require should be compatible with each other when symlinks are used

We looked at this issue in developer meeting today.

The ultimate reason why require and require_relative behaves differently is that while require_relative infers its argument's realpath every time, require doesn't.

This was by design; because require is called many times, we wanted to completely avoid disk access for 2nd and later calls to require with identical arguments.

But I believe the reported behaviour is a bug to be fixed. In order to do so a meeting attendee suggested to push both symlink-resolved and unresolved paths at once to $LOADED_FEATURES on the first call.

#6 [ruby-core:77581] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) almost 2 years ago

This could make it harder for auto-reloaders to unload a required file when require_relative is used... Doesn't seem like a great solution to this bug to me... Ruby could cache internally the real path when using "require" so that the second call would avoid any disk access...

#7 [ruby-core:77584] Updated by nobu (Nobuyoshi Nakada) almost 2 years ago

Shyouhei Urabe wrote:

In order to do so a meeting attendee suggested to push both symlink-resolved and unresolved paths at once to $LOADED_FEATURES on the first call.

I think this explanation differs from that we discussed accurately.
IIRC, it should be vm->expanded_load_path, not $LOADED_FEATURES.
$LOADED_FEATURES won't be doubled, but will have realpaths only.

#8 [ruby-core:77587] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) almost 2 years ago

Nobu, if a symlinked path would be removed from $LOADED_FEATURES from Ruby code, will all related internal references to the same file be cleared too?

#9 [ruby-core:77849] Updated by shyouhei (Shyouhei Urabe) over 1 year ago

Today I learned that PHP caches realpath, and causes troubles when people use symlink to deploy scripts.

The situation is not exactly the identical to ours, but we should avoid their footsteps.

#10 [ruby-core:82773] Updated by nobu (Nobuyoshi Nakada) 10 months ago

Would you expect just directory names get expanded?
Or basename too?
For instance, 'b loaded' should be printed twice or just once,?

mkdir a
echo "p 'b loaded'" > a/b.rb
ln -s b.rb a/c.rb
ruby -I./a -ra -rb -e''

#11 [ruby-core:82774] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) 10 months ago

I'd expect b.rb and c.rb to be handled like separate files, so "b loaded" would be printed twice.

#12 [ruby-core:82775] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) 10 months ago

I see, what you mean, if we simply add the expanded filenames to LOADED_PATH then that file would be loaded just once. Do you have any use cases where someone would symlink Ruby code for good usage?

#13 [ruby-core:82776] Updated by matsuda (Akira Matsuda) 10 months ago

Here's an actual use case that we saw in Rails: https://github.com/rails/rails/pull/29638#issuecomment-321335175
The reporter says that it happened in Jenkins, but I guess the same situation may happen in any case where we put the .rb files under a symlinked directory, for instance Capistrano.

#14 [ruby-core:82777] Updated by rosenfeld (Rodrigo Rosenfeld Rosas) 10 months ago

Akira, in those cases was the basename different among the real path and the symlink?

#15 Updated by nobu (Nobuyoshi Nakada) 10 months ago

  • Status changed from Open to Closed

Applied in changeset trunk|r59984.


load.c: real path to load

  • load.c (rb_construct_expanded_load_path): expand load paths to real paths to get rid of duplicate loading from symbolic-linked directories. [Feature #10222]

#16 Updated by nobu (Nobuyoshi Nakada) 6 months ago

  • Related to Bug #14372: Memory leak in require with Pathnames in the $LOAD_PATH in 2.3/2.4 added

#17 Updated by nagachika (Tomoyuki Chikanaga) 6 months ago

  • Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONTNEED

#18 [ruby-core:85275] Updated by nagachika (Tomoyuki Chikanaga) 6 months ago

I've filled Backport field according to the request at #14424.

#19 [ruby-core:85606] Updated by nagachika (Tomoyuki Chikanaga) 5 months ago

  • Backport changed from 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONTNEED to 2.3: REQUIRED, 2.4: DONE, 2.5: DONTNEED

ruby_2_4 r62440 merged revision(s) 59983,59984.

Also available in: Atom PDF