Feature #5999

Optimize for faster loading time and win32/file.c refactoring

Added by Hiroshi Shirosaki about 2 years ago. Updated about 2 years ago.

[ruby-core:42480]
Status:Closed
Priority:Normal
Assignee:Hiroshi Shirosaki
Category:core
Target version:2.0.0

Description

This was originally discussed at Best way to separate implementation specific code?.

Ruby startup time (loading time) is slow when loading a lot of libraries such as Rails. Some works were done to improve performance such as #3924, but startup is still not fast. Especially ruby on Windows seems much slower than on Unix like OS.
This work is an effort to improve startup time on Windows.

I have updated the patch with help from Luis Lavena. This patch improves loading time a little.

https://gist.github.com/1605263#file_0001_move_rb_file_load_ok_to_win32_file.c.patch

If loadpath and loadedfeatures are large numbers such as rails app, fileloadok() is called many times against non-existing path.

On Windows, GetFileAttributes() is faster than open(). And open() performance is affected by AntiVirus software's realtime scan.
I confirmed this with NOD32. Microsoft Security Essentials doesn't affect so badly as NOD32. It seems to depend on kinds of antivirus softwares.

Benchmark:
https://gist.github.com/1605263#gistcomment-75650
https://gist.github.com/1448652

Avoiding to call open() against non-existing path by using GetFileAttributes() seems to improve performance.
I used CreateFile() instead of open() because CreateFile() seems a little faster.

Here are rails startup benchmarks.

On Windows 7:

200ms improvement
https://gist.github.com/1605263#gistcomment-82137

On Windows XP:

800ms improvement with NOD32 OFF
1800ms improvement with NOD32 ON
https://gist.github.com/1605263#gistcomment-75644

This fix would be helpful for windows users. I guess most windows users use antivirus softwares.

I confirmed make, make test are OK. make test-all result was same as before patch at the following revision.
ruby 2.0.0dev (2012-02-09 trunk 34511) [i386-mingw32]

At , Matz agreed with replacing fileloadok.

This is the first work in win32/file.c. The following works (optimize expand_path etc.) into win32/file.c will be done by Luis.
This patch changed configure and Makefile. I also confirmed make with nmake(mswin32).

Is this acceptable way?

Associated revisions

Revision 34849
Added by shirosaki about 2 years ago

  • Makefile.in (PLATFORM_DIR): add a variable for win32 directory.
  • Makefile.in (clean-platform): add new target.
    It cleans win32 directory.

  • common.mk (clean): add a dependency for win32 directory.

  • common.mk (distclean): ditto.

  • common.mk (distclean-platform): add new target.
    It cleans win32 directory.

  • common.mk ($(PLATFORM_D)): add new target to make win32 directory.

  • common.mk (win32/win32.$(OBJEXT)): move win32.o into win32
    directory.

  • common.mk (win32/file.$(OBJEXT)): add new target for win32/file.c.

  • configure.in: move win32.o into win32 directory and add
    win32/file.o to MISSING.

  • file.c (fileloadok, rbfileloadok): replace static
    file
    loadok() with public rbfileloadok().
    It's to link Windows implementation in win32/file.c.

  • file.c (rbfindfileextsafe): ditto.

  • file.c (rbfindfile_safe): ditto.

  • win32/file.c (rbfileloadok): new file. Add Windows specific
    optimized implementation of rb
    fileloadok(). We created a
    separated file to avoid too many #ifdef macro which is unreadable.

  • win32/Makefile.sub (PLATFORM_DIR): add a variable for win32
    directory.

  • win32/Makefile.sub (MISSING): move win32.obj into win32
    directory and add win32/file.obj to MISSING.

  • win32/Makefile.sub (MAKEDIRS): replace MINIRUBY with BASERUBY.
    It's because miniruby doesn't exist when making win32 directory.

  • win32/Makefile.sub (clean-platform): add new target to clean win32
    directory.

  • win32/Makefile.sub ({$(srcdir)}.c{}.obj): make it not match
    win32/file.c to build properly.

  • win32/Makefile.sub (win32/win32.$(OBJEXT)): move win32.obj into
    win32 directory.

    Patch created with Luis Lavena.
    [Feature #5999]

History

#1 Updated by Luis Lavena about 2 years ago

  • Category set to core
  • Assignee set to Usaku NAKAMURA
  • Target version set to 2.0.0

#2 Updated by Luis Lavena about 2 years ago

Usa,

Do you approve this be merged?

Thank you.

#3 Updated by Jon Forums about 2 years ago

As there's been no negative feedback in the past 17 days, is this patch acceptable as-is and can it now be committed to trunk?

#4 Updated by Usaku NAKAMURA about 2 years ago

  • Status changed from Open to Assigned
  • Assignee changed from Usaku NAKAMURA to Hiroshi Shirosaki

I don't object.
Shirosaki-san, please commit it.

#5 Updated by Anonymous about 2 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 0 to 100

This issue was solved with changeset r34849.
Hiroshi, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.


  • Makefile.in (PLATFORM_DIR): add a variable for win32 directory.
  • Makefile.in (clean-platform): add new target.
    It cleans win32 directory.

  • common.mk (clean): add a dependency for win32 directory.

  • common.mk (distclean): ditto.

  • common.mk (distclean-platform): add new target.
    It cleans win32 directory.

  • common.mk ($(PLATFORM_D)): add new target to make win32 directory.

  • common.mk (win32/win32.$(OBJEXT)): move win32.o into win32
    directory.

  • common.mk (win32/file.$(OBJEXT)): add new target for win32/file.c.

  • configure.in: move win32.o into win32 directory and add
    win32/file.o to MISSING.

  • file.c (fileloadok, rbfileloadok): replace static
    file
    loadok() with public rbfileloadok().
    It's to link Windows implementation in win32/file.c.

  • file.c (rbfindfileextsafe): ditto.

  • file.c (rbfindfile_safe): ditto.

  • win32/file.c (rbfileloadok): new file. Add Windows specific
    optimized implementation of rb
    fileloadok(). We created a
    separated file to avoid too many #ifdef macro which is unreadable.

  • win32/Makefile.sub (PLATFORM_DIR): add a variable for win32
    directory.

  • win32/Makefile.sub (MISSING): move win32.obj into win32
    directory and add win32/file.obj to MISSING.

  • win32/Makefile.sub (MAKEDIRS): replace MINIRUBY with BASERUBY.
    It's because miniruby doesn't exist when making win32 directory.

  • win32/Makefile.sub (clean-platform): add new target to clean win32
    directory.

  • win32/Makefile.sub ({$(srcdir)}.c{}.obj): make it not match
    win32/file.c to build properly.

  • win32/Makefile.sub (win32/win32.$(OBJEXT)): move win32.obj into
    win32 directory.

    Patch created with Luis Lavena.
    [Feature #5999]

#6 Updated by Hiroshi Shirosaki about 2 years ago

I don't object.
Shirosaki-san, please commit it.

Thank you. I committed it at r34849.

--
Hiroshi Shirosaki

Also available in: Atom PDF