Bug #10222: require_relative and require should be compatible with each other when symlinks are used

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: 2.3.1, 2.1.2p95
• Backport: 2.3: REQUIRED, 2.4: DONE, 2.5: DONTNEED

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Not sure if this should be considered a bug or a feature request since I don't know whether the current behavior is intended or not.

Recently I got a report for my gem rails-web-console related to require_relative causing trouble with symlinked dirs:

https://github.com/rosenfeld/active_record_migrations/issues/6

Dmitry was able to replicate the issue using vanilla Ruby:

mkdir a
ln -s a b
echo "require_relative 'b'" > a/a.rb
echo "p 'b loaded'" > a/b.rb
echo "$: << File.expand_path('../b', __FILE__); require 'a'; require 'b'" > c.rb
ruby c.rb

Notice how "b loaded" is printed twice but if you replace require_relative with require it's just loaded once.

Shouldn't Ruby always expand the loaded files before appending them to the $LOADED_FEATURES and avoid this kind of error? I don't think require_relative should behave differently than a regular require in such cases.

Any thoughts?

[rosenfeld (Rodrigo Rosenfeld Rosas)]

I can't change the title myself. Could someone with privileges please change it to something like: "require_relative and require should be compatible with each other when symlinks are used".

I think this would make it easier to be searchable if others are experiencing the same issue. The key change is to add the "symlinks" word to the title so that the connection is made clear.

[shevegen (Robert A. Heiler)]

Curious behaviour indeed, there may be a reason why symlinks were assumed to behave differently.

[abolshakov (Tema Bolshakov)]

• ruby -v changed from 2.1.2p95 to 2.3.1

[abolshakov (Tema Bolshakov)]

• ruby -v changed from 2.3.1 to 2.3.1, 2.1.2p95

[shyouhei (Shyouhei Urabe)]

• Subject changed from require_relative and require should be compatible with each other to require_relative and require should be compatible with each other when symlinks are used

We looked at this issue in developer meeting today.

The ultimate reason why require and require_relative behaves differently is that while require_relative infers its argument's realpath every time, require doesn't.

This was by design; because require is called many times, we wanted to completely avoid disk access for 2nd and later calls to require with identical arguments.

But I believe the reported behaviour is a bug to be fixed. In order to do so a meeting attendee suggested to push both symlink-resolved and unresolved paths at once to $LOADED_FEATURES on the first call.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

This could make it harder for auto-reloaders to unload a required file when require_relative is used... Doesn't seem like a great solution to this bug to me... Ruby could cache internally the real path when using "require" so that the second call would avoid any disk access...

[nobu (Nobuyoshi Nakada)]

Shyouhei Urabe wrote:

In order to do so a meeting attendee suggested to push both symlink-resolved and unresolved paths at once to $LOADED_FEATURES on the first call.

I think this explanation differs from that we discussed accurately.
IIRC, it should be vm->expanded_load_path, not $LOADED_FEATURES.
$LOADED_FEATURES won't be doubled, but will have realpaths only.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Nobu, if a symlinked path would be removed from $LOADED_FEATURES from Ruby code, will all related internal references to the same file be cleared too?

[shyouhei (Shyouhei Urabe)]

Today I learned that PHP caches realpath, and causes troubles when people use symlink to deploy scripts.

The situation is not exactly the identical to ours, but we should avoid their footsteps.

[nobu (Nobuyoshi Nakada)]

Would you expect just directory names get expanded?
Or basename too?
For instance, 'b loaded' should be printed twice or just once,?

mkdir a
echo "p 'b loaded'" > a/b.rb
ln -s b.rb a/c.rb
ruby -I./a -ra -rb -e''

[rosenfeld (Rodrigo Rosenfeld Rosas)]

I'd expect b.rb and c.rb to be handled like separate files, so "b loaded" would be printed twice.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

I see, what you mean, if we simply add the expanded filenames to LOADED_PATH then that file would be loaded just once. Do you have any use cases where someone would symlink Ruby code for good usage?

[matsuda (Akira Matsuda)]

Here's an actual use case that we saw in Rails: https://github.com/rails/rails/pull/29638#issuecomment-321335175
The reporter says that it happened in Jenkins, but I guess the same situation may happen in any case where we put the .rb files under a symlinked directory, for instance Capistrano.

[rosenfeld (Rodrigo Rosenfeld Rosas)]

Akira, in those cases was the basename different among the real path and the symlink?

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset trunk|r59984.

---

load.c: real path to load

• load.c (rb_construct_expanded_load_path): expand load paths to
  real paths to get rid of duplicate loading from symbolic-linked
  directories. [Feature #10222]

[nobu (Nobuyoshi Nakada)]

• Related to Bug #14372: Memory leak in require with Pathnames in the $LOAD_PATH in 2.3/2.4 added

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN to 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONTNEED

[nagachika (Tomoyuki Chikanaga)]

I've filled Backport field according to the request at #14424.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.3: REQUIRED, 2.4: REQUIRED, 2.5: DONTNEED to 2.3: REQUIRED, 2.4: DONE, 2.5: DONTNEED

ruby_2_4 r62440 merged revision(s) 59983,59984.

[jeremyevans0 (Jeremy Evans)]

• Has duplicate Bug #13695: Issue with require and require_relative with symlinked directories added

[greneholt (Connor McKay)]

I recently encountered this issue on v2.7.1, when symlinked directories were included in the $RUBYLIB environment variable. I was not experiencing the issue on v2.5.1 with the exact same setup however. Has the bug regressed, or was $RUBYLIB never handled by this fix to begin with?

[mame (Yusuke Endoh)]

• Related to Feature #16978: Ruby should not use realpath for __FILE__ added

[shyouhei (Shyouhei Urabe)]

• Has duplicate Bug #17885: require_relative and require should be compatible with each other when symlinked files are used added