Project

General

Profile

Bug #11900

broken encoding of commandline on Windows

Added by usa (Usaku NAKAMURA) over 3 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
-
ruby -v:
ruby 2.4.0dev (2015-12-27 trunk 53345) [x64-mswin64_100]
[ruby-dev:49461]

Description

今頃気付いてしまったのですが、2.2以降のWindowsで、コマンドラインのエンコーディングが壊れています。
思い返すと、ちょうど1年前くらいになかださんがこの辺をいじっていたので、その時からではないかと。
trunkはもうめんどくさいからEncodingをUTF-8にしちゃっていい気がしますが、2.2と2.3はどうしたものすかね。

C:\>c:\ruby\2.1.8\bin\ruby -ve "s = 'てすと'; p [s, s.encoding, s.b]"
ruby 2.1.8p440 (2015-12-16 revision 53160) [x64-mswin64_100]
["てすと", #<Encoding:Windows-31J>, "\x82\xC4\x82\xB7\x82\xC6"]

C:\>c:\ruby\2.2.4\bin\ruby -ve "s = 'てすと'; p [s, s.encoding, s.b]"
ruby 2.2.4p230 (2015-12-16 revision 53155) [x64-mswin64_100]
["縺ヲ縺吶→", #<Encoding:Windows-31J>, "\xE3\x81\xA6\xE3\x81\x99\xE3\x81\xA8"]

C:\>c:\ruby\2.3.0\bin\ruby -ve "s = 'てすと'; p [s, s.encoding, s.b]"
ruby 2.3.0p0 (2015-12-25 revision 53290) [x64-mswin64_100]
["縺ヲ縺吶→", #<Encoding:Windows-31J>, "\xE3\x81\xA6\xE3\x81\x99\xE3\x81\xA8"]

C:\>ruby -ve "s = 'てすと'; p [s, s.encoding, s.b]"
ruby 2.4.0dev (2015-12-27 trunk 53345) [x64-mswin64_100]
["縺ヲ縺吶→", #<Encoding:Windows-31J>, "\xE3\x81\xA6\xE3\x81\x99\xE3\x81\xA8"]

Associated revisions

Revision 6ef87aa6
Added by nobu (Nobuyoshi Nakada) about 3 years ago

Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54785 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 54785
Added by nobu (Nobuyoshi Nakada) about 3 years ago

Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

Revision 54785
Added by nobu (Nobuyoshi Nakada) about 3 years ago

Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

Revision 54785
Added by nobu (Nobuyoshi Nakada) about 3 years ago

Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

Revision 54785
Added by nobu (Nobuyoshi Nakada) about 3 years ago

Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

Revision ce6fdd55
Added by nagachika (Tomoyuki Chikanaga) about 2 years ago

merge revision(s) 54785: [Backport #11900]

    * ruby.c (process_options): convert -e script to the encoding
      given by a command line option on Windows.  assume it is the
      expected encoding.  [ruby-dev:49461] [Bug #11900]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_3@58029 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 58029
Added by nagachika (Tomoyuki Chikanaga) about 2 years ago

merge revision(s) 54785: [Backport #11900]

* ruby.c (process_options): convert -e script to the encoding
  given by a command line option on Windows.  assume it is the
  expected encoding.  [ruby-dev:49461] [Bug #11900]

Revision bf2d4e93
Added by usa (Usaku NAKAMURA) about 2 years ago

merge revision(s) 54785: [Backport #11900]

    * ruby.c (process_options): convert -e script to the encoding
      given by a command line option on Windows.  assume it is the
      expected encoding.  [ruby-dev:49461] [Bug #11900]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/branches/ruby_2_2@58111 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Revision 58111
Added by usa (Usaku NAKAMURA) about 2 years ago

merge revision(s) 54785: [Backport #11900]

* ruby.c (process_options): convert -e script to the encoding
  given by a command line option on Windows.  assume it is the
  expected encoding.  [ruby-dev:49461] [Bug #11900]

History

Updated by usa (Usaku NAKAMURA) over 3 years ago

以下のパッチで直る(?)ような気がしますが、どうでしょうか>なかださん

Index: ruby.c
===================================================================
--- ruby.c  (リビジョン 53345)
+++ ruby.c  (作業コピー)
@@ -1535,6 +1535,9 @@ process_options(int argc, char **argv, struct cmdl
    }
    else {
        eenc = lenc;
+#if UTF8_PATH
+       opt->e_script = str_conv_enc(opt->e_script, rb_utf8_encoding(), eenc);
+#endif
    }
    rb_enc_associate(opt->e_script, eenc);
    if (!(opt->dump & ~DUMP_BIT(version_v))) {

Updated by usa (Usaku NAKAMURA) over 3 years ago

-Eが効いてなかったので訂正。

しかしこれだと-Uでも同等の効果が得られるわけですが、それが正しいのかどうか……。
-Uでは効かなくするにはこの修正箇所のちょっと上にあるif節の中でencをinternal encodingで上書きしている部分でencに代入しないようにする必要があります(一時変数として利用されてるだけなのでこの処理自体は本来は独立の気がする)。

Index: ruby.c
===================================================================
--- ruby.c  (リビジョン 53345)
+++ ruby.c  (作業コピー)
@@ -1534,7 +1534,10 @@ process_options(int argc, char **argv, struct cmdl
        eenc = rb_enc_from_index(opt->src.enc.index);
    }
    else {
-       eenc = lenc;
+       eenc = enc;
+#if UTF8_PATH
+       opt->e_script = str_conv_enc(opt->e_script, rb_utf8_encoding(), eenc);
+#endif
    }
    rb_enc_associate(opt->e_script, eenc);
    if (!(opt->dump & ~DUMP_BIT(version_v))) {

Updated by naruse (Yui NARUSE) about 3 years ago

Usaku NAKAMURA wrote:

しかしこれだと-Uでも同等の効果が得られるわけですが、それが正しいのかどうか……。
-Uでは効かなくするにはこの修正箇所のちょっと上にあるif節の中でencをinternal encodingで上書きしている部分でencに代入しないようにする必要があります(一時変数として利用されてるだけなのでこの処理自体は本来は独立の気がする)。

-U は -e で与えたソースのエンコーディングにも影響するはずなので、通常SJISな-eソースも、-Uが与えられていればUTF-8になるはずですね。
なお、現在のパッチだと-Ksを指定した場合には壊れたままになる気がするので、そこも直す必要があります。

Updated by usa (Usaku NAKAMURA) about 3 years ago

入れてみましょう。

#6

Updated by nobu (Nobuyoshi Nakada) about 3 years ago

  • Status changed from Assigned to Closed

Applied in changeset r54785.


Fix -e script encoding

  • ruby.c (process_options): convert -e script to the encoding given by a command line option on Windows. assume it is the expected encoding. [ruby-dev:49461] [Bug #11900]

Updated by nagachika (Tomoyuki Chikanaga) about 2 years ago

  • Backport changed from 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED, 2.3: REQUIRED to 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED, 2.3: DONE

ruby_2_3 r58029 merged revision(s) 54785.

Updated by usa (Usaku NAKAMURA) about 2 years ago

  • Backport changed from 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: REQUIRED, 2.3: DONE to 2.0.0: DONTNEED, 2.1: DONTNEED, 2.2: DONE, 2.3: DONE

ruby_2_2 r58111 merged revision(s) 54785.

Also available in: Atom PDF