https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112009-07-19T07:13:26ZRuby Issue Tracking SystemRuby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=48692009-07-19T07:13:26Znormalperson (Eric Wong)normalperson@yhbt.net
<ul></ul><p>=begin<br>
This patch totally breaks UNIX applications that rely on this behavior. It's<br>
widely considered good practice to unlink temp files ASAP on platforms<br>
that support it for several reasons:</p>
<ol>
<li>
<p>reduced chance of conflicts/retries when other temp files are created</p>
</li>
<li>
<p>less chance a fatal error in the application causing disk space to<br>
be filled up because finalizers didn't get run</p>
</li>
<li>
<p>improved security in case an accidental chmod hits the file, or<br>
the user is running another misbehaving application.</p>
</li>
</ol>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=48722009-07-19T08:37:20Zniels (Niels Ganser)niels@herimedia.com
<ul></ul><p>=begin<br>
In fact, back in 2004, only seconds after posting his patch, Florian already "came to the conclusion that this patch sucks": <a href="http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/2849?2697-2915+split-mode-vertical" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/2849?2697-2915+split-mode-vertical</a></p>
<p>Matz chimed in shortly thereafter and consensus seemed to be to simply rescue (and discard) EACCESS as the file will be garbage collected after unlinking anyways: <a href="http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/2850?2697-2915+split-mode-vertical" class="external">http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-core/2850?2697-2915+split-mode-vertical</a></p>
<p>As Eric noted above, the current implementation goes contrary to what many UNIX programmers have learned for years.</p>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53532009-08-21T06:24:37Zbitsweat (Jeremy Daer)jeremydaer@gmail.com
<ul></ul><p>=begin<br>
Please revert this change. It's causing major issues for everyone BUT windows.</p>
<p><a href="http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4" class="external">http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4</a></p>
<p>Please restore the previous, sane behavior and come up a different, non-regressive fix for the original concern.</p>
<p>It's bad enough, and there's no response to the issue, to fork the tempfile stdlib: <a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28</a><br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53542009-08-21T06:32:47Zluislavena (Luis Lavena)luislavena@gmail.com
<ul></ul><p>=begin<br>
On Thu, Aug 20, 2009 at 6:24 PM, Jeremy Kemper<a href="mailto:redmine@ruby-lang.org" class="email">redmine@ruby-lang.org</a> wrote:</p>
<blockquote>
<p>Issue <a class="issue tracker-1 status-8 priority-4 priority-default closed" title="Bug: tempfile#unlink may silently fail on windows (Third Party's Issue)" href="https://bugs.ruby-lang.org/issues/1494">#1494</a> has been updated by Jeremy Kemper.</p>
<p>Please revert this change. It's causing major issues for everyone BUT windows.</p>
<p><a href="http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4" class="external">http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4</a></p>
<p>Please restore the previous, sane behavior and come up a different, non-regressive fix for the original concern.</p>
<p>It's bad enough, and there's no response to the issue, to fork the tempfile stdlib: <a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28</a></p>
</blockquote>
<p>Sorry to chime in, but why when it rescue the Errno::EACCES, it<br>
doesn't close and then retry? Instead of just silently fail.</p>
<p><a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L336" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L336</a></p>
<p>If the error is known, and the workaround is also known, why we should<br>
keep rescuing this in our when when the library is just plainly<br>
ignoring it?</p>
<h2>--<br>
Luis Lavena<br>
AREA 17</h2>
<p>Perfection in design is achieved not when there is nothing more to add,<br>
but rather when there is nothing more to take away.<br>
Antoine de Saint-Exupéry</p>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53552009-08-21T07:36:20Zshyouhei (Shyouhei Urabe)shyouhei@ruby-lang.org
<ul><li><strong>Category</strong> set to <i>lib</i></li><li><strong>Assignee</strong> set to <i>matz (Yukihiro Matsumoto)</i></li><li><strong>Priority</strong> changed from <i>3</i> to <i>5</i></li><li><strong>Target version</strong> set to <i>1.9.1</i></li></ul><p>=begin<br>
Seems it's (currently) 1.9.x specific so I moved this issue to 1.9 category.</p>
<p>I've also assigned it to matz because revision r23494 was checed in by him.</p>
<p>Matz, can I revert this? Or do you want to do so?<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53562009-08-21T08:37:48Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>=begin<br>
Hi,</p>
<p>At Fri, 21 Aug 2009 07:36:27 +0900,<br>
Shyouhei Urabe wrote in <a href="https://blade.ruby-lang.org/ruby-core/25008">[ruby-core:25008]</a>:</p>
<blockquote>
<p>Seems it's (currently) 1.9.x specific so I moved this issue to 1.9 category.</p>
<p>I've also assigned it to matz because revision r23494 was checed in by him.</p>
<p>Matz, can I revert this? Or do you want to do so?</p>
</blockquote>
<p>I'd vote for Luis's suggestion.</p>
<p>--<br>
Nobu Nakada</p>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53582009-08-21T08:53:07Zmatz (Yukihiro Matsumoto)matz@ruby.or.jp
<ul></ul><p>=begin<br>
Hi,</p>
<p>In message "Re: <a href="https://blade.ruby-lang.org/ruby-core/25011">[ruby-core:25011]</a> Re: [Bug <a class="issue tracker-1 status-8 priority-4 priority-default closed" title="Bug: tempfile#unlink may silently fail on windows (Third Party's Issue)" href="https://bugs.ruby-lang.org/issues/1494">#1494</a>] tempfile#unlink may silently fail on windows"<br>
on Fri, 21 Aug 2009 08:37:35 +0900, Nobuyoshi Nakada <a href="mailto:nobu@ruby-lang.org" class="email">nobu@ruby-lang.org</a> writes:</p>
<p>|At Fri, 21 Aug 2009 07:36:27 +0900,<br>
|Shyouhei Urabe wrote in <a href="https://blade.ruby-lang.org/ruby-core/25008">[ruby-core:25008]</a>:<br>
|> Seems it's (currently) 1.9.x specific so I moved this issue to 1.9 category.<br>
|><br>
|> I've also assigned it to matz because revision r23494 was checed in by him.<br>
|><br>
|> Matz, can I revert this? Or do you want to do so?<br>
|<br>
|I'd vote for Luis's suggestion.</p>
<p>I am not sure what he exactly wants, and what would happen on Windows.<br>
But if you think it's OK, please check in the fix.</p>
<pre><code> matz.
</code></pre>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53592009-08-21T10:41:57Zusa (Usaku NAKAMURA)usa@garbagecollect.jp
<ul></ul><p>=begin<br>
To begin with, What situation did the original post encount?<br>
I doubt that they tried to check the existence of the file after unlink.</p>
<p>About Tempfile, it is nonsense to check the existence.<br>
There are another proper ways only if they wants the path name to check whether there is a directory-entry on the filesystem or not.</p>
<p>In conclusion, I recommend to revert the patch.</p>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53622009-08-21T17:04:39Zluislavena (Luis Lavena)luislavena@gmail.com
<ul></ul><p>=begin<br>
On Thu, Aug 20, 2009 at 8:52 PM, Yukihiro Matsumoto<a href="mailto:matz@ruby-lang.org" class="email">matz@ruby-lang.org</a> wrote:</p>
<blockquote>
<p>Hi,</p>
<p>In message "Re: <a href="https://blade.ruby-lang.org/ruby-core/25011">[ruby-core:25011]</a> Re: [Bug <a class="issue tracker-1 status-8 priority-4 priority-default closed" title="Bug: tempfile#unlink may silently fail on windows (Third Party's Issue)" href="https://bugs.ruby-lang.org/issues/1494">#1494</a>] tempfile#unlink may silently fail on windows"<br>
on Fri, 21 Aug 2009 08:37:35 +0900, Nobuyoshi Nakada <a href="mailto:nobu@ruby-lang.org" class="email">nobu@ruby-lang.org</a> writes:</p>
<p>|At Fri, 21 Aug 2009 07:36:27 +0900,<br>
|Shyouhei Urabe wrote in <a href="https://blade.ruby-lang.org/ruby-core/25008">[ruby-core:25008]</a>:<br>
|><br>
|> Matz, can I revert this? Or do you want to do so?<br>
|<br>
|I'd vote for Luis's suggestion.</p>
<p>I am not sure what he exactly wants, and what would happen on Windows.<br>
But if you think it's OK, please check in the fix.</p>
</blockquote>
<p>Hello Matz,</p>
<p>Basically as Jeremy Kemper pointed, the current 1.9.x implementation is flawed.</p>
<p>On this thread:<br>
<a href="http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4" class="external">http://groups.google.com/group/rack-devel/browse_thread/thread/a2aab3a4720f34c4</a></p>
<p>A workaround is use "better" external dependency to override Ruby's<br>
one, which has been mentioned in the above thread and it's on GitHub:</p>
<p><a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L28</a></p>
<p>More precisely, the approach shown here:</p>
<p><a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L328-340" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L328-340</a></p>
<p>Seems to rescue Errno::EACCES on Windows, and silently fail.</p>
<p>On our scenario (Windows), that will leave open descriptors.</p>
<p>My suggestion was: if we are now taking the time to rescue that<br>
exception why we don't actually solve the issue?</p>
<pre><code> rescue Errno::EACCES
close
retry
end
</code></pre>
<p>In theory that will reduce lot of Errno:EACCES and platform specific<br>
conditions in end-users scripts.</p>
<p>The only thing under this could fail is if another process has locked<br>
that file too.</p>
<h2>--<br>
Luis Lavena<br>
AREA 17</h2>
<p>Perfection in design is achieved not when there is nothing more to add,<br>
but rather when there is nothing more to take away.<br>
Antoine de Saint-Exupéry</p>
<p>=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53632009-08-21T17:43:47Zhongli (Hongli Lai)hongli@phusion.nl
<ul></ul><p>=begin<br>
Hi guys. Let me re-explain the issue:</p>
<p>There are times when you just need a large anonymous disk-backed byte buffer. For example, when you're writing a web server and you want to buffer the client's upload data. In cases like this, the filename of the buffer temp file you're using doesn't matter, it just matters that you have a file handle open; you're not going to pass the filename to another process or anything.</p>
<p>POSIX systems allow you to do just this. You create a file, unlink it from the filesystem, but keep the file handle open. The result is a file handle that only you can access - no other processes can access it because there's no filesystem entry anymore. This doesn't work on Microsoft Windows however, because on Windows one may only unlink a file if nobody has it open.</p>
<p>Portable applications that need a disk-backed byte buffer will want to do something like this on POSIX:</p>
<ol>
<li>Create the file.</li>
<li>Unlink it. This will always succeed.</li>
<li>Use the file handle to do things.</li>
<li>When done, close the file handle.</li>
</ol>
<p>On Windows on the other hand they'd want to do this instead:</p>
<ol>
<li>Create the file.</li>
<li>Use the file handle to do things.</li>
<li>When done, close the file handle.</li>
<li>Unlink the file.</li>
</ol>
<p>Tempfile can support this in a portable manner by recommending applications to use it as follows:</p>
<ol>
<li>Create the file.</li>
<li>Unlink it if possible (i.e. only on POSIX systems).</li>
<li>Use the file handle to do things. It doesn't matter whether the previous unlink action succeeded, but the file handle must remain open because the app is going to do some work with it.</li>
<li>When done, close the file handle.</li>
<li>If the unlink action in #2 failed, then unlink it now.</li>
</ol>
<p>Making the #unlink call close the file handle is not acceptable; it will totally destroy step #3. When calling #unlink, the programmer does not want to close it; it just wants the filesystem entry to disappear, if platform allows it, but the file handle must remain open.</p>
<p>Part of the problem is that not everybody knows this trick or that POSIX and Windows behave differently. The current Tempfile documentation documents this very sparsely.</p>
<p>My version of Tempfile:</p>
<ul>
<li>Supports the aforementioned use case: <a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L307-327" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L307-327</a>
</li>
<li>Has extensive documentation which, among other things, explains the unlink-after-closing trick and all the cross-platform caveats that come with it.<br>
<a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L307-327" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L307-327</a> and<br>
<a href="http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L76-91" class="external">http://github.com/FooBarWidget/better/blob/master/lib/better/tempfile.rb#L76-91</a>
</li>
<li>Has extensive documentation for everything else as well.</li>
<li>Fixes a few bugs. These are explained in the 'Comparison' section in the overview documentation of the class.</li>
<li>Has many unit tests. Ruby's Tempfile currently has almost no unit tests.</li>
</ul>
<p>Please consider accepting it upstream.<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=53682009-08-22T06:07:12Zhongli (Hongli Lai)hongli@phusion.nl
<ul></ul><p>=begin<br>
Luis Lavena:</p>
<blockquote>
<p>Seems to rescue Errno::EACCES on Windows, and silently fail.</p>
</blockquote>
<p>Actually, Ruby 1.9's Tempfile already does that. I just removed the 'close' call in the #unlink method. I'd rather advocate letting the exception propagate and explicitly documenting the exception in the API documentation, but I want to preserve Tempfile's current behavior as much as possible so I preserved that code and introduced #unlinked? instead.<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=54012009-08-25T17:55:15Zhongli (Hongli Lai)hongli@phusion.nl
<ul></ul><p>=begin<br>
Any updates on this?<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=54142009-08-26T13:49:55Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Third Party's Issue</i></li><li><strong>Assignee</strong> deleted (<del><i>matz (Yukihiro Matsumoto)</i></del>)</li></ul><p>=begin<br>
RubyInline's issue.<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=54152009-08-26T14:26:30Znaruse (Yui NARUSE)naruse@airemix.jp
<ul></ul><p>=begin<br>
The revert commit is r24662.<br>
Yugui will backport this to 1.9.1.</p>
<p>If you have some enhancement request, create new ticket per issue and attach patches for ruby's trunk.<br>
=end</p> Ruby master - Bug #1494: tempfile#unlink may silently fail on windowshttps://bugs.ruby-lang.org/issues/1494?journal_id=54172009-08-26T17:17:00Zhongli (Hongli Lai)hongli@phusion.nl
<ul></ul><p>=begin<br>
Great, thanks for reverting.</p>
<p>I've opened an enhancement request for my Tempfile improvements: <a href="http://redmine.ruby-lang.org/issues/show/1999" class="external">http://redmine.ruby-lang.org/issues/show/1999</a><br>
=end</p>