Project

General

Profile

Actions

Feature #6845

closed

Optimize stat on Windows

Added by h.shirosaki (Hiroshi Shirosaki) about 12 years ago. Updated about 12 years ago.

Status:
Closed
Target version:
[ruby-core:47083]

Description

require slowness on startup was pointed out and some improvements were done, but startup seems still slow especially on Windows.
I suggest a patch to optimize require(stat) performance on Windows.

https://gist.github.com/3285446#file_1_winstat.patch

This patch was originally created by Dušan D. Majkić. I reviewed it and added some fixes.

Rationale:

When using require, stat() is called in realpath_rec(). Currently FindFirstFile() is used for stat() implementation on Windows.
According to our benchmark, GetFileAttributesEx() is the fastest way to get file attributes.

We changed stat implementation to use GetFileAttributesEx() and fallback to FindFirstFile().
GetFileAttributesEx() has possibility to get ERROR_SHARING_VIOLATION. For example against 'C:\pagefile.sys'.[1]
In such cases, we use FindFirstFile() to get file info. I added a test for this.

check_valid_dir() in winnt_stat() was introduced due to the following issue.
http://bugs.ruby-lang.org/issues/1005

This function uses FindFirstFile() after GetFileAttributes() to check if the path is valid directory or not.
This is for checking paths like "...". So I limited FindFirstFile() call only if the path contains "..." for performance.

Benchmark:

I can see small performance improvement with this patch. Here are benchmark results on Windows 7.
I used measurements [2] and empty rails app.

https://gist.github.com/3285446#file_2_bench.md
https://gist.github.com/3285446#gistcomment-393221 (chart)

Notable improvement is this.

core_require_nested 3.41s to 2.96s (-0.45s) (13% faster)
(require of deeply nested path)

One behavior difference:

This patch changes stat result of root path. Trunk ruby doesn't get root path's time information. Times are zero.
But patched ruby can get times of root path as expected. It's because FindFirstFile() cannot get root attributes.

trunk

C:\Users\hiroshi\work>ruby -e "p File.stat('/')"
#<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0, rdev=0x2, size=0, blksize=nil, bloc
ks=nil, atime=1970-01-01 09:00:00 +0900, mtime=1970-01-01 09:00:00 +0900, ctime=1970-01-01 09:00:00
+0900>

patched

C:\Users\hiroshi\work>ruby -e "p File.stat('/')"
#<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0, rdev=0x2, size=0, blksize=nil, bloc
ks=nil, atime=2012-08-07 21:55:09 +0900, mtime=2012-08-07 21:55:09 +0900, ctime=2009-07-14 11:38:56
+0900>

I confirmed make test and make test-all. It seems fine.

Also Luis Lavena, I and other people are testing TCS ruby 1.9.3 which includes Dušan's patch [3] and didn't see any issues while several months.

Thank you.

[1] http://hg.openjdk.java.net/icedtea/jdk7/jdk/rev/e2d9696aa701
[2] https://github.com/jonforums/measurements
[3] https://github.com/thecodeshop/ruby/commits/winstat/ruby_1_9_3


Files

winnt_stat.patch (3.7 KB) winnt_stat.patch h.shirosaki (Hiroshi Shirosaki), 08/09/2012 07:46 AM

Related issues 1 (0 open1 closed)

Related to Backport193 - Backport #6846: Please backport Feature #6845Closed08/10/2012Actions

Updated by usa (Usaku NAKAMURA) about 12 years ago

  • Category set to core
  • Status changed from Open to Assigned
  • Assignee set to h.shirosaki (Hiroshi Shirosaki)
  • Target version set to 2.0.0

seems good.
commit it, please.

Actions #2

Updated by Anonymous about 12 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r36668.
Hiroshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


Optimize winnt_stat

  • test/ruby/test_file_exhaustive.rb
    (TestFileExhaustive#test_stat_special_file): add a test.
    GetFileAttributesExW fails to get attributes of special files
    such as pagefile.sys.

  • win32/win32.c (check_valid_dir): for performance, check the path
    by FindFirstFileW only if the path containts "..."

  • win32/win32.c (winnt_stat): use GetFileAttributesExW instead of
    FindFirstFileW since GetFileAttributesExW is faster.
    Based on the patch by Dusan D. Majkic.
    [ruby-core:47083] [Feature #6845]

Actions

Also available in: Atom PDF

Like0
Like0Like0