Project

General

Profile

Bug #11444

STDIN.each_codepoint hangs indefinitely on certain codepoints

Added by cantino (Andrew Cantino) over 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
ruby -v:
ruby 2.3.0dev (2015-08-13 trunk 51569) [x86_64-linux]
[ruby-core:<unknown>]

Description

Piping certain Unicode data into Ruby and calling each_codepoint on STDIN results in a hang. Control-c does not exit, but control-z will. I've tested this in 2.2.0, 2.2.2, and trunk.

Please see https://gist.github.com/cantino/48f977bee6a39e33f80f for reproduction. There is an example file with various Unicode Emoji characters in it (borrowed from http://apps.timwhitlock.info/emoji/tables/unicode).

I'd like to help fix this if anyone can point me in the right direction. I fiddled with the file until I found a state where it works if one character is removed from the front of the file, so hopefully that's useful information.


Related issues

Related to Ruby master - Feature #1667: IO#codepoints, IO#each_codepoint, and StringIOClosedActions

Associated revisions

Revision cfa7550b
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51583 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 51583
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

Revision 51583
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

Revision 51583
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

Revision 51583
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

Revision 51583
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]

Revision 84bf320b
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51594 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 51594
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

Revision 51594
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

Revision 51594
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

Revision 51594
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

Revision 51594
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: raise at incomplete char

  • io.c (rb_io_each_codepoint): raise an exception at incomplete character before EOF when conversion takes place. [Bug #11444]

Revision 0bf77265
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 51638
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

Revision 51638
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

Revision 51638
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

Revision 51638
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

Revision 51638
Added by nobu (Nobuyoshi Nakada) over 4 years ago

io.c: initialize variable

  • io.c (rb_io_each_codepoint): fix use of uninitialized variable. [Bug #11444]

Revision 6d8ed07d
Added by nagachika (Tomoyuki Chikanaga) almost 4 years ago

merge revision(s) 51583,51638: [Backport #11444]

    * io.c (rb_io_each_codepoint): read more data when read partially.
      [ruby-core:70379] [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@52786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 52786
Added by nagachika (Tomoyuki Chikanaga) almost 4 years ago

merge revision(s) 51583,51638: [Backport #11444]

* io.c (rb_io_each_codepoint): read more data when read partially.
  [ruby-core:70379] [Bug #11444]

Revision 4af525b1
Added by nagachika (Tomoyuki Chikanaga) almost 4 years ago

merge revision(s) 51594: [Backport #11444]

    * io.c (rb_io_each_codepoint): raise an exception at incomplete
      character before EOF when conversion takes place.  [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@52804 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 52804
Added by nagachika (Tomoyuki Chikanaga) almost 4 years ago

merge revision(s) 51594: [Backport #11444]

* io.c (rb_io_each_codepoint): raise an exception at incomplete
  character before EOF when conversion takes place.  [Bug #11444]

Revision f81b6694
Added by usa (Usaku NAKAMURA) almost 4 years ago

merge revision(s) 51583,51594,51638: [Backport #11444]

    * io.c (rb_io_each_codepoint): read more data when read partially.
      [ruby-core:70379] [Bug #11444]

    * io.c (rb_io_each_codepoint): raise an exception at incomplete
      character before EOF when conversion takes place.  [Bug #11444]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_1@52831 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 52831
Added by usa (Usaku NAKAMURA) almost 4 years ago

merge revision(s) 51583,51594,51638: [Backport #11444]

* io.c (rb_io_each_codepoint): read more data when read partially.
  [ruby-core:70379] [Bug #11444]

* io.c (rb_io_each_codepoint): raise an exception at incomplete
  character before EOF when conversion takes place.  [Bug #11444]

History

#1

Updated by nobu (Nobuyoshi Nakada) over 4 years ago

  • Description updated (diff)
  • Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED
#2

Updated by nobu (Nobuyoshi Nakada) over 4 years ago

  • Status changed from Open to Closed

Applied in changeset r51583.


io.c: read more data

  • io.c (rb_io_each_codepoint): read more data when read partially. [ruby-core:70379] [Bug #11444]
#3

Updated by cantino (Andrew Cantino) over 4 years ago

Thank you for fixing this, and in less than 24 hours!

#4

Updated by nobu (Nobuyoshi Nakada) over 4 years ago

  • Related to Feature #1667: IO#codepoints, IO#each_codepoint, and StringIO added

Updated by nagachika (Tomoyuki Chikanaga) almost 4 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE

Backported r51583 and r51638 into ruby_2_2 at r52786.
I didn't backport r51594 because it introduce an incompatibility.

Updated by usa (Usaku NAKAMURA) almost 4 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED

Tomoyuki Chikanaga wrote:

I didn't backport r51594 because it introduce an incompatibility.

But without r51549, the test never pass on mswin.
See http://ruby-mswin.cloudapp.net/vc10-x64/ruby-2.2/log/20151129T090947Z.log.html.gz#rubyspec

Updated by nagachika (Tomoyuki Chikanaga) almost 4 years ago

usa san: Thank you for your notice.
Hmm, I will backport r51594.

Updated by nagachika (Tomoyuki Chikanaga) almost 4 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE

Backported r51594 into ruby_2_2 branch at r52804.

Updated by usa (Usaku NAKAMURA) almost 4 years ago

  • Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE to 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONE

ruby_2_1 r52831 merged revision(s) 51583,51594,51638.

Also available in: Atom PDF