Bug #22528: rb_enc_cr_str_buf_cat frees the buffer with the wrong size when an empty string adopts a wide encoding

• Status: Open
• Assignee: -
• Target version: -
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

[kddnewton (Kevin Newton)]

When a UTF-16 or UTF-32 string is appended to an empty string whose buffer has no room for the wider terminator, the old buffer is freed with a size computed from the new terminator length.

s = ("a" * 1000).encode("UTF-16LE")
String.new(capacity: 2000, encoding: "UTF-8") << s
# [BUG] buffer 0x... freed with old_size=2002, but was allocated with size=2001

rb_enc_cr_str_buf_cat sets the new encoding before it adjusts the terminator:

rb_str_buf_cat(str, ptr, len);
ENCODING_CODERANGE_SET(str, ptr_encindex, ptr_cr);
rb_str_change_terminator_length(str, rb_enc_mbminlen(str_enc), rb_enc_mbminlen(ptr_enc));

If rb_str_change_terminator_length has to reallocate, str_make_independent_expand frees the old buffer with STR_HEAP_SIZE(str), and the TERM_LEN(str) in that macro already reflects the new encoding. In the interpolation example the buffer has a capacity of 4001 and was allocated with a 1-byte terminator (4002 bytes), but it is freed as 4001 + 4 (4005 bytes).

I believe this is the second half of https://bugs.ruby-lang.org/issues/22519.

cc @mame (Yusuke Endoh)