Project

General

Profile

Actions

Feature #6845

closed

Optimize stat on Windows

Added by h.shirosaki (Hiroshi Shirosaki) over 11 years ago. Updated over 11 years ago.

Status:
Closed
Target version:
[ruby-core:47083]

Description

require slowness on startup was pointed out and some improvements were done, but startup seems still slow especially on Windows.
I suggest a patch to optimize require(stat) performance on Windows.

https://gist.github.com/3285446#file_1_winstat.patch

This patch was originally created by Dušan D. Majkić. I reviewed it and added some fixes.

Rationale:

When using require, stat() is called in realpath_rec(). Currently FindFirstFile() is used for stat() implementation on Windows.
According to our benchmark, GetFileAttributesEx() is the fastest way to get file attributes.

We changed stat implementation to use GetFileAttributesEx() and fallback to FindFirstFile().
GetFileAttributesEx() has possibility to get ERROR_SHARING_VIOLATION. For example against 'C:\pagefile.sys'.[1]
In such cases, we use FindFirstFile() to get file info. I added a test for this.

check_valid_dir() in winnt_stat() was introduced due to the following issue.
http://bugs.ruby-lang.org/issues/1005

This function uses FindFirstFile() after GetFileAttributes() to check if the path is valid directory or not.
This is for checking paths like "...". So I limited FindFirstFile() call only if the path contains "..." for performance.

Benchmark:

I can see small performance improvement with this patch. Here are benchmark results on Windows 7.
I used measurements [2] and empty rails app.

https://gist.github.com/3285446#file_2_bench.md
https://gist.github.com/3285446#gistcomment-393221 (chart)

Notable improvement is this.

core_require_nested 3.41s to 2.96s (-0.45s) (13% faster)
(require of deeply nested path)

One behavior difference:

This patch changes stat result of root path. Trunk ruby doesn't get root path's time information. Times are zero.
But patched ruby can get times of root path as expected. It's because FindFirstFile() cannot get root attributes.

trunk

C:\Users\hiroshi\work>ruby -e "p File.stat('/')"
#<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0, rdev=0x2, size=0, blksize=nil, bloc
ks=nil, atime=1970-01-01 09:00:00 +0900, mtime=1970-01-01 09:00:00 +0900, ctime=1970-01-01 09:00:00
+0900>

patched

C:\Users\hiroshi\work>ruby -e "p File.stat('/')"
#<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0, rdev=0x2, size=0, blksize=nil, bloc
ks=nil, atime=2012-08-07 21:55:09 +0900, mtime=2012-08-07 21:55:09 +0900, ctime=2009-07-14 11:38:56
+0900>

I confirmed make test and make test-all. It seems fine.

Also Luis Lavena, I and other people are testing TCS ruby 1.9.3 which includes Dušan's patch [3] and didn't see any issues while several months.

Thank you.

[1] http://hg.openjdk.java.net/icedtea/jdk7/jdk/rev/e2d9696aa701
[2] https://github.com/jonforums/measurements
[3] https://github.com/thecodeshop/ruby/commits/winstat/ruby_1_9_3


Files

winnt_stat.patch (3.7 KB) winnt_stat.patch h.shirosaki (Hiroshi Shirosaki), 08/09/2012 07:46 AM

Related issues 1 (0 open1 closed)

Related to Backport193 - Backport #6846: Please backport Feature #6845Closed08/10/2012Actions
Actions

Also available in: Atom PDF

Like0
Like0Like0