Bug #17308: RubyVM::InstructionSequence.compile_file doesn't use the same default encoding than Kernel.load

• Status: Closed
• Assignee: -
• Target version: -
• Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN

[byroot (Jean Boussier)]

  def test_compile_file_encoding
    Tempfile.create(%w"test_iseq .rb") do |f|
      f.puts '{ "Þ" => "Th", "ß" => "ss", "à" => "a" }'
      f.close

      previous_external = Encoding.default_external
      Encoding.default_external = Encoding::US_ASCII
      begin
        load f.path
        RubyVM::InstructionSequence.compile_file(f.path)
      ensure
        Encoding.default_external = previous_external
      end
    end
  end

In the above test case, load assumes that the source file is UTF-8 whereas compile_file uses default_external causing various problems:

[1/1] TestISeq#test_compile_file_encoding test_iseq20201106-96756-2c0cz.rb:1: warning: key "" is duplicated and overwritten on line 1
test_iseq20201106-96756-2c0cz.rb:1: warning: key "" is duplicated and overwritten on line 1
 = 0.00 s
  1) Error:
TestISeq#test_compile_file_encoding:
SyntaxError: test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
test_iseq20201106-96756-2c0cz.rb:1: invalid multibyte char (US-ASCII)
    ruby/test/ruby/test_iseq.rb:283:in `compile_file'
    ruby/test/ruby/test_iseq.rb:283:in `block in test_compile_file_encoding'
    ruby/lib/tempfile.rb:358:in `create'
    ruby/test/ruby/test_iseq.rb:275:in `test_compile_file_encoding'

I believe compile_file should be consistent with load, or at least properly default to UTF-8 like load does.

[jeremyevans0 (Jeremy Evans)]

I submitted a pull request to make RubyVM::InstructionSequence.compile_file use same encoding as Kernel#load: https://github.com/ruby/ruby/pull/3788

• Status changed from Open to Closed

Applied in changeset git|4a5c42db88d30532bd4fbcdff89615ebf961d2a2.

---

Make RubyVM::InstructionSequence.compile_file use same encoding as load

This switches the internal function from rb_parser_compile_file_path
to rb_parser_load_file, which is the same internal method that
Kernel#load uses.

Fixes [Bug #17308]