Bug #22519: Incompatibility between parse.y and Prism in the encoding of interpolated strings

• Status: Closed
• Assignee: prism
• Target version: -
• Backport: 3.3: UNKNOWN, 3.4: UNKNOWN, 4.0: UNKNOWN

[mame (Yusuke Endoh)]

There seems to be an incompatibility between parse.y and Prism in the encoding of interpolated strings.

# in a script file
x = 1
p "#{x}#{x}".encoding  # parse.y: #<Encoding:UTF-8>  prism: #<Encoding:US-ASCII>

a = "abc".force_encoding("US-ASCII")
b = "".encode("UTF-16LE")
s = "#{a}#{b}"
p s.encoding           # parse.y: #<Encoding:UTF-8>  prism: #<Encoding:UTF-16LE>
p s                    # parse.y: "abc"              prism: "\u6261\x63"

@naruse (Yui NARUSE) says the behavior of parse.y is correct. This is probably a bug in the Prism compiler, but I have not looked into it in detail.

With -e, the result of parse.y seems to depend on the locale encoding (the LANG environment variable). I have not looked into the details of this either.

[byroot (Jean Boussier)]

Yes, Parse.y puts an extra empty string with the final encoding:

0003 putobject                              ""
0005 getlocal_WC_0                          x@0
0007 dup
0008 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0010 anytostring
0011 getlocal_WC_0                          x@0
0013 dup
0014 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0016 anytostring
0017 concatstrings                          3

Whereas prism elude it:

0003 getlocal_WC_0                          x@0
0005 dup
0006 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0008 anytostring
0009 getlocal_WC_0                          x@0
0011 dup
0012 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
0014 anytostring
0015 concatstrings                          2

[Earlopain (Earlopain _)]

I found https://github.com/ruby/ruby/pull/11728. I'm guessing that introduced this behaviour, it used to always emit the empty string.

[kddnewton (Kevin Newton)]

https://github.com/ruby/ruby/pull/19815

[kddnewton (Kevin Newton)]

• Status changed from Open to Closed

Applied in changeset git|7a07e456e4bc5e68dbe56cf943fa0ef93d34016a.

---

Fix up interpolation in prism compiler

When an interpolation is the first element of a string and the
interpolated values have different encodings, we need an empty
string at the start to get the negotiation correct.

[Bug #22519]