Bug #11444: STDIN.each_codepoint hangs indefinitely on certain codepoints

• Status: Closed
• Assignee: -
• Target version: -
• ruby -v: ruby 2.3.0dev (2015-08-13 trunk 51569) [x86_64-linux]
• Backport: 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONE

[cantino (Andrew Cantino)]

Piping certain Unicode data into Ruby and calling each_codepoint on STDIN results in a hang. Control-c does not exit, but control-z will. I've tested this in 2.2.0, 2.2.2, and trunk.

Please see https://gist.github.com/cantino/48f977bee6a39e33f80f for reproduction. There is an example file with various Unicode Emoji characters in it (borrowed from http://apps.timwhitlock.info/emoji/tables/unicode).

I'd like to help fix this if anyone can point me in the right direction. I fiddled with the file until I found a state where it works if one character is removed from the front of the file, so hopefully that's useful information.

[nobu (Nobuyoshi Nakada)]

• Description updated (diff)
• Backport changed from 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED

[nobu (Nobuyoshi Nakada)]

• Status changed from Open to Closed

Applied in changeset r51583.

---

io.c: read more data

• io.c (rb_io_each_codepoint): read more data when read partially.
  [ruby-core:70379] [Bug #11444]

[cantino (Andrew Cantino)]

Thank you for fixing this, and in less than 24 hours!

[nobu (Nobuyoshi Nakada)]

• Related to Feature #1667: IO#codepoints, IO#each_codepoint, and StringIO added

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE

Backported r51583 and r51638 into ruby_2_2 at r52786.
I didn't backport r51594 because it introduce an incompatibility.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED

Tomoyuki Chikanaga wrote:

I didn't backport r51594 because it introduce an incompatibility.

But without r51549, the test never pass on mswin.
See http://ruby-mswin.cloudapp.net/vc10-x64/ruby-2.2/log/20151129T090947Z.log.html.gz#rubyspec

[nagachika (Tomoyuki Chikanaga)]

usa san: Thank you for your notice.
Hmm, I will backport r51594.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: REQUIRED to 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE

Backported r51594 into ruby_2_2 branch at r52804.

[usa (Usaku NAKAMURA)]

• Backport changed from 2.0.0: REQUIRED, 2.1: REQUIRED, 2.2: DONE to 2.0.0: REQUIRED, 2.1: DONE, 2.2: DONE

ruby_2_1 r52831 merged revision(s) 51583,51594,51638.