Bug #12551: Exception accessing file with long path on windows

• Status: Closed
• Priority: Normal
• Assignee: -
• Target version: -
• ruby -v: ruby 2.3.0p0 (2015-12-25 revision 53290) [x64-mingw32]
• Backport: 2.1: UNKNOWN, 2.2: UNKNOWN, 2.3: UNKNOWN
• Tags: core, win

[Maturin (Lars Benner)]

Accessing file with a long file name causes this exception:

C:/Projects/Head/Build/cm/testdir01/testdir02/testdir03/testdir04/testdir05/testdir06/testdir07/testdir08/testdir09/testdir10/testdir11/testdir12/testdir13/testdir14/testdir15/testdir16/testdir17/testdir18/testdir19/testdir20/testdir21/testdir22/longtestfile.name.txt
repro_lstat_exception.rb:42:in stat': No such file or directory @ rb_file_s_stat - C:/Projects/Head/Build/cm/testdir01/testdir02/testdir03/testdir04/testdir05/testdir06/testdir07/testdir08/testdir09/testdir10/testdir11/testdir12/testdir13/testdir14/testdir15/testdir16/testdir17/testdir18/testdir19/testdir20/testdir21/testdir22/longtestfile.name.txt (Errno::ENOENT) from repro_lstat_exception.rb:42:in block in '
from repro_lstat_exception.rb:13:in chdir' from repro_lstat_exception.rb:13:in '

Can be reproduced with the attach script.

I used windows 7 64 bit and an NTFS file system.

For me it looks like, it has something to do with the legacy MAX_PATH_LENGTH of 260 characters. See https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath for more information.

Originally I discovered the bug with JRuby (https://github.com/jruby/jruby/issues/3995).

With older Ruby version, I tried 1.8.5, it is working.

[headius (Charles Nutter)]

FYI, the JRuby issue is waiting on a decision from MRI about how to handle this.

[shyouhei (Shyouhei Urabe)]

• Status changed from Open to Assigned

[usa (Usaku NAKAMURA)]

Sorry for late reply.

It's too difficult to get rid of the limitation of MAX_PATH,
because many OS API and system functions in C runtime depend on it.

OS API can handle long path with \\?\ prefix, but when using this prefix,
the rest must be an absolute path.
Ruby cannot guarantee that a path passed by a user is an absolute path,
so we cannot use this trick at all times.

[nobu (Nobuyoshi Nakada)]

• Has duplicate Bug #14631: Add support for long paths in Windows 10 when support is enabled in the OS added

[preetpalS (Preetpal Sohal)]

usa (Usaku NAKAMURA) wrote:

It's too difficult to get rid of the limitation of MAX_PATH,
because many OS API and system functions in C runtime depend on it.

OS API can handle long path with \\?\ prefix, but when using this prefix,
the rest must be an absolute path.
Ruby cannot guarantee that a path passed by a user is an absolute path,
so we cannot use this trick at all times.

Python 3.6.2 (64 bits) is able to handle really long paths even when supplied relative paths. I'm not sure how Python is able to do this but its approach might be relevant to solving this issue. Also note that the MAX_PATH limitation was removed from common Win32 file and directory functions starting with Windows 10, Version 1607 (reference: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath).

import os

i = 1
pathToCreate = "python_long_relative_path_stress_test\\" + str(i)
while i < 1000:
  try:
    i = i + 1
    pathToCreate = pathToCreate + "\\" + str(i)
    os.makedirs(pathToCreate)
  except:
    print("Failed to create folder:")
    print(len(os.path.abspath(pathToCreate)))
    exit(1)

print("Could create path of size (using relative paths):")
print(len(os.path.abspath(pathToCreate)))

[shevegen (Robert A. Heiler)]

I guess if python can do it, ruby will support it as well eventually. :-)

I wish I could help more but my C knowledge is so limited compared
to my knowledge in ruby (and I use linix almost all of the time
rather than windows).

[matz (Yukihiro Matsumoto)]

• Status changed from Assigned to Open
• Assignee deleted (usa (Usaku NAKAMURA))

As Python can do it, we know it is theoretically possible. But the priority is low for us, core developers. If someone sends us a pull-request, we'd be happy to review.

Matz.

[gabuscus (Gabriel Nagy)]

Hello,

I managed to compile Ruby with Visual C++ and long paths support by using an application manifest. I validated some basic things like executing a script from a really long path, and creating a directory with path over 255 characters.

This is the code, which works, but I'm hardcoding stuff since I don't know where would be the best place to put it: https://github.com/GabrielNagy/ruby/commit/eb785c9a567ce058b7e2f16d3a1f53014f245687

I'd appreciate if someone could point me to how I could implement this in a cleaner way.

Thanks!

[nobu (Nobuyoshi Nakada)]

Thank you for the input.
It seems different from this document a little:
https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later
Is it convertible?

[gabuscus (Gabriel Nagy)]

I changed it to match the example from Microsoft:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1" xmlns:asmv3="urn:schemas-microsoft-com:asm.v3">
  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
      <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
  </application>
</assembly>

The assembly tag is required as per https://docs.microsoft.com/en-us/windows/win32/sbscs/application-manifests

[nobu (Nobuyoshi Nakada)]

Does this work?
https://github.com/ruby/ruby/compare/master...nobu:bug/12551-win32-long-path

[gabuscus (Gabriel Nagy)]

Almost! :D Thank you for the makefile improvements.

This took me way longer than expected but I finally have it working: https://github.com/ruby/ruby/pull/4505

[gabuscus (Gabriel Nagy)]

• Status changed from Open to Closed

Applied in changeset git|229cb0fcdb7957b19d7042b000d803ae58cc6593.

---

[Win32] long path name support [Bug #12551]

Implement long path support on Windows by applying Microsoft's
recommended application manifest.

To make this work on both Visual C++ and MinGW, include the manifest as
a resource when generating the resource files. This way it will be
embedded into the executables generated by both compilers.

It's important for the manifest resource to have ID 1, otherwise GCC
will embed a default manifest.

Note that in addition to this, the user needs to have long paths enabled
either by modifying the registry or by enabling a group policy.

Co-authored-by: Nobuyoshi Nakada [e-mail]