Project

General

Profile

Actions

Bug #7211

closed

eval does not respect encoding magic comment

Added by vo.x (Vit Ondruch) over 11 years ago. Updated over 11 years ago.

Status:
Rejected
Assignee:
-
Target version:
-
ruby -v:
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-linux]
Backport:
[ruby-core:48188]

Description

=begin
This should work IMO.

$ LANG=C irb
irb(main):001:0> eval <<EOF
irb(main):002:0" # encoding: utf-8
irb(main):003:0" puts "vít"
irb(main):004:0" EOF
SyntaxError: (irb):3: invalid multibyte char (US-ASCII)
(irb):1: syntax error, unexpected $end, expecting tSTRING_CONTENT or tSTRING_DBEG or tSTRING_DVAR or tSTRING_END
from /usr/bin/irb:12:in `'

I am currently bitten by this behavior in RubyGems. I have .gemspec [1], which have some UTF-8 characters, with the encoding magic comment. While building gem, RubyGems evals the .gemspec code [1], where the magic comment is ignored and build fails:

$ LANG=C gem build gem-nice-install.gemspec
ERROR: While executing gem ... (ArgumentError)
invalid byte sequence in US-ASCII

[1] https://gist.github.com/3944821

[2] https://github.com/rubygems/rubygems/blob/master/lib/rubygems/specification.rb#L907
=end

Updated by headius (Charles Nutter) over 11 years ago

Under typical circumstances, isn't the string already going to have an encoding? Even in IRB, it's using the console default encoding, so the String does have an inherent encoding already.

The magic comment is used by the parser to know what encoding to assign to an otherwise opaque stream of bytes coming from a script. That's not the case for eval.

I'm not saying it wouldn't be useful, but it would basically amount to transcoding the eval'ed string to that encoding if it's different than the String's natural encoding, or else ignoring it completely if they're the same. In other words:

coding: UTF-8

eval "# coding: ASCII-8BIT\nputs 'blah'"

would behave identically to:

eval "puts 'blah'".encode('ASCII-8BIT')

and

coding: UTF-8

eval "# coding: UTF-8\nputs 'blah'"

would do nothing with encoding at all...

Updated by nobu (Nobuyoshi Nakada) over 11 years ago

  • Status changed from Open to Rejected

It fails before eval, but in IRB input.

Updated by drbrain (Eric Hodel) over 11 years ago

=begin
Even without irb the description does not make sense.

The code used on line 907 is read as UTF-8 on line 899, so the following would be the equivalent and works correctly:

$ LANG=C ruby19 -ve 'eval File.read("t.rb", mode: "r:UTF-8:-")'
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-darwin12.2.0]
π
$ cat t.rb
puts 'π'
$ LANG=C ruby19 -ve 'eval File.read("t.rb", mode: "r:UTF-8:-")'
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-darwin12.2.0]
π

Note that reading without UTF-8 results in a SyntaxError, not an ArgumentError.

$ LANG=C ruby19 -ve 'eval File.read("t.rb")'
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-darwin12.2.0]
-e:1:in eval': (eval):1: invalid multibyte char (US-ASCII) (SyntaxError) (eval):1: invalid multibyte char (US-ASCII) from -e:1:in '

I think you will need to file a bug against RubyGems including the backtrace and your gemspec: gem build --backtrace …

=end

Actions

Also available in: Atom PDF

Like0
Like0Like0Like0