https://bugs.ruby-lang.org/
https://bugs.ruby-lang.org/favicon.ico?1711330511
2008-11-03T02:34:38Z
Ruby Issue Tracking System
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1513
2008-11-03T02:34:38Z
gnufied (hemant kumar)
gethemant@gmail.com
<ul></ul><p>=begin<br>
I do not see this problem, with:</p>
<p>ruby 1.9.0 (2008-09-30 revision 0) [i686-linux]</p>
<p>=end</p>
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1515
2008-11-03T09:06:09Z
matz (Yukihiro Matsumoto)
matz@ruby.or.jp
<ul></ul><p>=begin<br>
Hi,</p>
<p>In message "Re: <a href="/issues/703">[ruby-core:19668]</a> [Bug <a class="issue tracker-1 status-5 priority-4 priority-default closed" title="Bug: string output duplication occurs if the same file descriptor written to in different threads (Closed)" href="https://bugs.ruby-lang.org/issues/703">#703</a>] string output duplication occurs if the same file descriptor written to in different threads"<br>
on Sat, 1 Nov 2008 02:38:52 +0900, Roger Pack <a href="mailto:redmine@ruby-lang.org" class="email">redmine@ruby-lang.org</a> writes:</p>
<p>|ruby-dev:32566</p>
<p>It's not related to ruby-dev:32566, I think.</p>
<p>|a = Thread.new { p '4'}<br>
|b = Thread.new { p '3' }<br>
|a.join<br>
|b.join<br>
|<br>
|results in<br>
|"4"<br>
|"3"<br>
|"4"<br>
|"3"<br>
|<br>
|because the buffer is modified simultaneously by two threads within io_fflush [I think].</p>
<p>It doesn't happen on either trunk nor 1.8.7. What's your version, and<br>
platform?</p>
<pre><code> matz.
</code></pre>
<p>=end</p>
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1518
2008-11-03T09:46:14Z
nobu (Nobuyoshi Nakada)
nobu@ruby-lang.org
<ul></ul><p>=begin<br>
Hi,</p>
<p>At Mon, 3 Nov 2008 09:04:37 +0900,<br>
Yukihiro Matsumoto wrote in <a href="https://blade.ruby-lang.org/ruby-core/19676">[ruby-core:19676]</a>:</p>
<blockquote>
<p>It doesn't happen on either trunk nor 1.8.7. What's your version, and<br>
platform?</p>
</blockquote>
<p>It could reproduced with full ruby, but not with miniruby nor<br>
full ruby with disabling gems.</p>
<p>$ ./ruby -v bug-703.rb<br>
ruby 1.9.0 (2008-11-03 revision 20092) [i686-linux]<br>
"4"<br>
"3"<br>
"4"<br>
"3"</p>
<p>$ ./ruby --disable-gems bug-703.rb<br>
"4"<br>
"3"</p>
<p>Interestingly, -v makes it occur again.</p>
<p>$ ./ruby -v --disable-gems bug-703.rb<br>
ruby 1.9.0 (2008-11-03 revision 20092) [i686-linux]<br>
"4"<br>
"3"<br>
"4"<br>
"3"</p>
<p>But -v after --disable-gems doesn't.</p>
<p>$ ./ruby --disable-gems -v bug-703.rb<br>
ruby 1.9.0 (2008-11-03 revision 20092) [i686-linux]<br>
"4"<br>
"3"</p>
<p>--<br>
Nobu Nakada</p>
<p>=end</p>
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1523
2008-11-04T02:46:13Z
nobu (Nobuyoshi Nakada)
nobu@ruby-lang.org
<ul></ul><p>=begin<br>
Hi,</p>
<p>At Mon, 3 Nov 2008 09:44:44 +0900,<br>
Nobuyoshi Nakada wrote in <a href="https://blade.ruby-lang.org/ruby-core/19678">[ruby-core:19678]</a>:</p>
<blockquote>
<blockquote>
<p>It doesn't happen on either trunk nor 1.8.7. What's your version, and<br>
platform?</p>
</blockquote>
<p>It could reproduced with full ruby, but not with miniruby nor<br>
full ruby with disabling gems.</p>
</blockquote>
<p>It seems a mere timing and probability issue. And seems solved<br>
by serializing write operations, as Roger figured out.</p>
<pre><code class="patch syntaxhl" data-language="patch"><span class="gh">Index: gc.c
===================================================================
</span><span class="gd">--- gc.c (revision 20101)
</span><span class="gi">+++ gc.c (working copy)
</span><span class="p">@@ -1509,4 +1509,5 @@</span> gc_mark_children(rb_objspace_t *objspace
gc_mark(objspace, obj->as.file.fptr->writeconv_pre_ecopts, lev);
gc_mark(objspace, obj->as.file.fptr->encs.ecopts, lev);
<span class="gi">+ gc_mark(objspace, obj->as.file.fptr->write_lock, lev);
</span> }
break;
<span class="gh">Index: io.c
===================================================================
</span><span class="gd">--- io.c (revision 20101)
</span><span class="gi">+++ io.c (working copy)
</span><span class="p">@@ -525,9 +525,27 @@</span> rb_write_internal(int fd, void *buf, siz
}
<span class="gi">+static long
+io_writable_length(rb_io_t *fptr, long l)
+{
+ if (PIPE_BUF < l &&
+ !rb_thread_alone() &&
+ wsplit_p(fptr)) {
+ l = PIPE_BUF;
+ }
+ return l;
+}
+
+static VALUE
+io_flush_buffer(VALUE arg)
+{
+ rb_io_t *fptr = (rb_io_t *)arg;
+ long l = io_writable_length(fptr, fptr->wbuf_len);
+ return rb_write_internal(fptr->fd, fptr->wbuf+fptr->wbuf_off, l);
+}
+
</span> static int
io_fflush(rb_io_t *fptr)
{
<span class="gd">- int r, l;
- int wbuf_off, wbuf_len;
</span><span class="gi">+ long r;
</span>
rb_io_check_closed(fptr);
<span class="p">@@ -540,13 +558,5 @@</span> io_fflush(rb_io_t *fptr)
if (fptr->wbuf_len == 0)
return 0;
<span class="gd">- wbuf_off = fptr->wbuf_off;
- wbuf_len = fptr->wbuf_len;
- l = wbuf_len;
- if (PIPE_BUF < l &&
- !rb_thread_alone() &&
- wsplit_p(fptr)) {
- l = PIPE_BUF;
- }
- r = rb_write_internal(fptr->fd, fptr->wbuf+wbuf_off, l);
</span><span class="gi">+ r = rb_mutex_synchronize(fptr->write_lock, io_flush_buffer, (VALUE)fptr);
</span> /* xxx: Other threads may modify wbuf.
* A lock is required, definitely. */
<span class="p">@@ -732,9 +742,23 @@</span> make_writeconv(rb_io_t *fptr)
/* writing functions */
<span class="gi">+struct binwrite_arg {
+ rb_io_t *fptr;
+ VALUE str;
+ long offset;
+ long length;
+};
+
+static VALUE
+io_binwrite_string(VALUE arg)
+{
+ struct binwrite_arg *p = (struct binwrite_arg *)arg;
+ long l = io_writable_length(p->fptr, p->length);
+ return rb_write_internal(p->fptr->fd, RSTRING_PTR(p->str)+p->offset, l);
+}
</span>
static long
io_binwrite(VALUE str, rb_io_t *fptr, int nosync)
{
<span class="gd">- long len, n, r, l, offset = 0;
</span><span class="gi">+ long len, n, r, offset = 0;
</span>
len = RSTRING_LEN(str);
<span class="p">@@ -745,7 +769,10 @@</span> io_binwrite(VALUE str, rb_io_t *fptr, in
fptr->wbuf_capa = 8192;
fptr->wbuf = ALLOC_N(char, fptr->wbuf_capa);
<span class="gi">+ fptr->write_lock = rb_mutex_new();
</span> }
if ((!nosync && (fptr->mode & (FMODE_SYNC|FMODE_TTY))) ||
(fptr->wbuf && fptr->wbuf_capa <= fptr->wbuf_len + len)) {
<span class="gi">+ struct binwrite_arg arg;
+
</span> /* xxx: use writev to avoid double write if available */
if (fptr->wbuf_len && fptr->wbuf_len+len <= fptr->wbuf_capa) {
<span class="p">@@ -767,12 +794,10 @@</span> io_binwrite(VALUE str, rb_io_t *fptr, in
rb_io_check_closed(fptr);
}
<span class="gi">+ arg.fptr = fptr;
+ arg.str = str;
+ arg.offset = offset;
</span> retry:
<span class="gd">- l = n;
- if (PIPE_BUF < l &&
- !rb_thread_alone() &&
- wsplit_p(fptr)) {
- l = PIPE_BUF;
- }
- r = rb_write_internal(fptr->fd, RSTRING_PTR(str)+offset, l);
</span><span class="gi">+ arg.length = n;
+ r = rb_mutex_synchronize(fptr->write_lock, io_binwrite_string, (VALUE)&arg);
</span> /* xxx: other threads may modify given string. */
if (r == n) return len;
<span class="p">@@ -3040,4 +3065,17 @@</span> finish_writeconv(rb_io_t *fptr, int nora
}
<span class="gi">+struct finish_writeconv_arg {
+ rb_io_t *fptr;
+ int noraise;
+};
+
+static VALUE
+finish_writeconv_sync(VALUE arg)
+{
+ struct finish_writeconv_arg *p = (struct finish_writeconv_arg *)arg;
+ finish_writeconv(p->fptr, p->noraise);
+ return Qnil;
+}
+
</span> static void
fptr_finalize(rb_io_t *fptr, int noraise)
<span class="p">@@ -3045,5 +3083,13 @@</span> fptr_finalize(rb_io_t *fptr, int noraise
int ebadf = 0;
if (fptr->writeconv) {
<span class="gd">- finish_writeconv(fptr, noraise);
</span><span class="gi">+ if (fptr->write_lock) {
+ struct finish_writeconv_arg arg;
+ arg.fptr = fptr;
+ arg.noraise = noraise;
+ rb_mutex_synchronize(fptr->write_lock, finish_writeconv_sync, (VALUE)&arg);
+ }
+ else {
+ finish_writeconv(fptr, noraise);
+ }
</span> }
if (fptr->wbuf_len) {
<span class="gh">Index: include/ruby/io.h
===================================================================
</span><span class="gd">--- include/ruby/io.h (revision 20101)
</span><span class="gi">+++ include/ruby/io.h (working copy)
</span><span class="p">@@ -74,4 +74,5 @@</span> typedef struct rb_io_t {
int writeconv_initialized;
<span class="gi">+ VALUE write_lock;
</span> } rb_io_t;
<span class="p">@@ -134,4 +135,5 @@</span> typedef struct rb_io_t {
fp->encs.ecflags = 0;\
fp->encs.ecopts = Qnil;\
<span class="gi">+ fp->write_lock = 0;\
</span> } while (0)
<span class="err">
</span></code></pre>
<p>--<br>
Nobu Nakada</p>
<p>=end</p>
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1546
2008-11-08T05:47:07Z
nobu (Nobuyoshi Nakada)
nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Closed</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>=begin<br>
Applied in changeset r20144.<br>
=end</p>
Ruby master - Bug #703: string output duplication occurs if the same file descriptor written to in different threads
https://bugs.ruby-lang.org/issues/703?journal_id=1586
2008-11-11T14:38:13Z
rogerdpack (Roger Pack)
rogerpack2005@gmail.com
<ul></ul><p>=begin<br>
Thank you that fixed it. I don't have OS X to be able to tell if the test named after ruby-dev:32566 is also fixed but wouldn't be surprised if it was.<br>
Regards.<br>
-=R<br>
=end</p>