Bug #13034: rb_enc_ascget() does not handle 'needmore' error from rb_enc_procise_mbclen()

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.1: UNKNOWN, 2.2: DONE, 2.3: DONE

[rhenium (Kazuki Yamaguchi)]

rb_enc_ascget() stores the return value from rb_enc_procise_mbclen() that may return a negative value on error to 'unsigned int', and thus it overlooks errors due to incomplete UTF-16/32 byte sequence.

This bug is visible from Ruby; for example, the following code gets stuck in infinite loop:

str = "\x00".force_encoding("UTF-16BE")
Regexp.quote(str) # Ruby freezes

• Status changed from Open to Closed

Applied in changeset r57078.

---

encoding.c: handle needmore error from rb_enc_precise_mbclen()

rb_enc_ascget() erroneously reports success even if the given byte
sequence is incomplete, for non-ASCII compatible encoding strings.

rb_enc_precise_mbclen() may return a negative value on error, and thus
rb_enc_ascget() must not store the return value in 'unsigned int';
otherwise the subsequent MBCLEN_CHARFOUND_P() check won't catch the
error. [ruby-core:78646] [Bug #13034]

[usa (Usaku NAKAMURA)]

• Backport changed from 2.1: UNKNOWN, 2.2: REQUIRED, 2.3: REQUIRED to 2.1: UNKNOWN, 2.2: DONE, 2.3: REQUIRED

ruby_2_2 r57217 merged revision(s) 57078.

[nagachika (Tomoyuki Chikanaga)]

• Backport changed from 2.1: UNKNOWN, 2.2: DONE, 2.3: REQUIRED to 2.1: UNKNOWN, 2.2: DONE, 2.3: DONE

ruby_2_3 r57348 merged revision(s) 57078.