https://bugs.ruby-lang.org/https://bugs.ruby-lang.org/favicon.ico?17113305112023-01-26T23:03:25ZRuby Issue Tracking SystemRuby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1014972023-01-26T23:03:25Zaustin (Austin Ziegler)halostatue@gmail.com
<ul></ul><p>It’s been a long time since I’ve used Windows, but the Windows console is notoriously stuck in 1980s encodings and using codepage 65001 <em>should</em> fix this in general. Otherwise, you’re going to get Windows 1252 encoding as your default input/output encoding even if Ruby is otherwise using UTF-8.</p>
<p>I believe that since Ruby 3.0, Ruby by default uses UTF-8 but the boundaries caused by your console codepage may be a confounding factor.</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1014982023-01-26T23:59:21Zstringsn88keys (Thomas Powell)
<ul></ul><p>By "console" do you mean irb are you referencing PowerShell or cmd.exe/Command Prompt? Windows Terminal produces the same results as well.</p>
<p>Also, the source for this is from one process to another without user interactivity.</p>
<p>Looking at the Code Page 437 vs. Windows-1252, 0xE4 would be ∑ in Code Page 437 and ä in Windows-1252</p>
<p>The byte sequence of "Mitteleuropäische Zeit" as encoded from <code>Time.now.zone</code> (which reports itself as "IBM437" is (hex values):<br>
<code>=> ["4d", "69", "74", "74", "65", "6c", "65", "75", "72", "6f", "70", "e4", "69", "73", "63", "68", "65", "20", "5a", "65", "69", "74"]</code></p>
<p>70 e4 69 would be "päi" in Windows-1252, but "p∑i" in IBM437 as reported. If UTF-8 is assumed, then e4 is a leading byte for a CJK script byte, but packing them doesn't associate the e4 with the following byte, which is confirmed by occasional invalid byte sequence errors depending on how the string is picked up.</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015052023-01-27T02:49:39Zaustin (Austin Ziegler)halostatue@gmail.com
<ul></ul><p>Yes, I mean cmd.exe or any other windows command-line. I repeat, that it has been <em>years</em> since I have used Windows in any serious manner, but this was the absolute <em>bane</em> of my existence, and process <em>boundaries</em> on Windows were nightmarish when I last dealt with Windows, primarily because of the emphasis on backwards compatibility at any cost. There does appear to be a bug if <code>Time.now.zone</code> is not returning the data in the code page expected for where it’s used (e.g., UTF-8).</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015072023-01-27T07:08:42Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Open</i> to <i>Feedback</i></li></ul><p>What are:</p>
<ul>
<li>the output from <code>chcp.com</code> command</li>
<li><code>Encoding.locale_charmap</code></li>
</ul>
<p>?</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015172023-01-27T13:22:06Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>Maybe msvcrt converts timezone names to ACP, not ConsoleCP.<br>
If so, this patch may work, but I have no idea how to test this in a CI.</p>
<pre><code class="diff syntaxhl" data-language="diff"><span class="gh">diff --git i/time.c w/time.c
index 9c4c93939e0..2e1a2dca29b 100644
</span><span class="gd">--- i/time.c
</span><span class="gi">+++ w/time.c
</span><span class="p">@@ -929,7 +929,7 @@</span> timegmw_noleapsecond(struct vtm *vtm)
}
static VALUE
<span class="gd">-zone_str(const char *zone)
</span><span class="gi">+zone_str_enc(const char *zone, rb_encoding *enc)
</span> {
const char *p;
int ascii_only = 1;
<span class="p">@@ -950,11 +950,18 @@</span> zone_str(const char *zone)
str = rb_usascii_str_new(zone, len);
}
else {
<span class="gd">- str = rb_enc_str_new(zone, len, rb_locale_encoding());
</span><span class="gi">+ if (!enc) enc = rb_locale_encoding();
+ str = rb_enc_str_new(zone, len, enc);
</span> }
return rb_fstring(str);
}
<span class="gi">+static VALUE
+zone_str(const char *zone)
+{
+ return zone_str_enc(zone, NULL);
+}
+
</span> static void
gmtimew_noleapsecond(wideval_t timew, struct vtm *vtm)
{
<span class="p">@@ -1653,12 +1660,18 @@</span> localtime_with_gmtoff_zone(const time_t *t, struct tm *result, long *gmtoff, VAL
#if defined(HAVE_TM_ZONE)
*zone = zone_str(tm.tm_zone);
#elif defined(HAVE_TZNAME) && defined(HAVE_DAYLIGHT)
<span class="gi">+ rb_encoding *enc = NULL;
+# if defined(_WIN32)
+ char cp[(sizeof(UINT) * 8 / 3) + 4];
+ snprintf(cp, sizeof(cp), "CP%u", GetACP());
+ enc = rb_enc_find(cp);
+# endif
</span> # if defined(RUBY_MSVCRT_VERSION) && RUBY_MSVCRT_VERSION >= 140
# define tzname _tzname
# define daylight _daylight
# endif
/* this needs tzset or localtime, instead of localtime_r */
<span class="gd">- *zone = zone_str(tzname[daylight && tm.tm_isdst]);
</span><span class="gi">+ *zone = zone_str_enc(tzname[daylight && tm.tm_isdst], enc);
</span> #else
{
char buf[64];
</code></pre> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015182023-01-27T13:48:52Zstringsn88keys (Thomas Powell)
<ul></ul><p>nobu (Nobuyoshi Nakada) wrote in <a href="#note-4">#note-4</a>:</p>
<blockquote>
<p>What are:</p>
<ul>
<li>the output from <code>chcp.com</code> command</li>
<li><code>Encoding.locale_charmap</code></li>
</ul>
<p>?</p>
</blockquote>
<p>chcp.com output:<br>
<code>"Aktive Codepage: 437."</code> (<code>"Active code page: 437"</code> on English display language.)</p>
<p>Encoding.locale_charmap # => "CP437" (German and English)</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015282023-01-27T21:05:14Zstringsn88keys (Thomas Powell)
<ul></ul><p>The top level status of this bug says "Closed" but last updated status says "Feedback". Can anyone clarify?</p> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1015442023-01-30T02:23:00Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Assigned</i></li><li><strong>Assignee</strong> set to <i>windows</i></li></ul> Ruby master - Bug #19383: Time.now.zone encoding for German display language in Windows is incorrecthttps://bugs.ruby-lang.org/issues/19383?journal_id=1021222023-03-03T02:15:34Znobu (Nobuyoshi Nakada)nobu@ruby-lang.org
<ul></ul><p>How about the patch at <a href="#note-5">#note-5</a>?</p>