Project

General

Profile

Actions

Feature #10085

closed

Add non-ASCII case conversion to String#upcase/downcase/swapcase/capitalize

Added by duerst (Martin Dürst) over 9 years ago. Updated over 2 years ago.

Status:
Closed
Target version:
-
[ruby-core:63964]

Description

Case conversion functions are currently limited to ASCII characters. When used with formal languages, that may be appropriate, but it is often not appropriate for applications.

In order to avoid backwards-compatibility problems and to make sure that the various variants of case conversion (e.g. language-dependent) can be selected, we propose to add an optional parameter to the case conversion functions.

Our current design idea is as follows:

ASCII-only if no parameter:
'Türkiye'.upcase # => 'TüRKIYE', note lower-case ü

Parameter triggers (general) Unicode conversion:
'Türkiye'.upcase 'en' # => 'TÜRKIYE', note upper-case Ü

The parameter is actually a BCP 47 (http://tools.ietf.org/html/bcp47) language tag.
This means that for languages with special case conversion rules, such as Turkish, this works as follows:
'Türkiye'.upcase 'tr' # => 'TÜRKİYE', note upper-case İ (with dot!)

In the second example, we used 'en', but most other languages would work, too, because a single case conversion works for most languages. Turkic languages are the biggest exception.

The Unicode standard also defines various cases of "case-folding", which usually is lossy, e.g. mapping German ß to
ss and so on. It should be possible to include this functionality in this proposal, e.g. by using :symbols or CONSTANTs for the few specific foldings. It may also be possible to define a reversible variant of case conversion in particular for use with swapcase.

In the long term, instead of a direct BCP 47 string, we could create a Locale class that would incorporate language-specific facilities, but this may need more detailed considerations.

The idea of using an additional parameter to indicate language-dependent or other processing variants should be extensible to areas such as number-to-string conversion and date formation. While this proposal is only about case conversion, we should check that there is a good chance to use similar parameter conventions for such extensions.

[This proposal is based on research done together with my student Kimihito Matsui.]


Files

CaseConversion.pdf (340 KB) CaseConversion.pdf duerst (Martin Dürst), 07/23/2014 11:05 AM

Related issues 7 (2 open5 closed)

Related to Ruby master - Bug #3376: russian supportClosednaruse (Yui NARUSE)06/01/2010Actions
Related to Ruby master - Feature #2034: Consider the ICU Library for Improving and Expanding Unicode SupportRejectednaruse (Yui NARUSE)Actions
Related to Ruby master - Feature #10002: String swapcaseClosedActions
Related to Ruby master - Feature #10152: String#strip doesn't remove non-breaking spaceOpen08/19/2014Actions
Related to Ruby master - Bug #10550: Resolv::DNS.getaddresses returns no IPs when nameserver returns in differing case than queryClosed11/26/2014Actions
Related to Ruby master - Feature #19317: Unicode ICU Full case mappingAssignedduerst (Martin Dürst)Actions
Has duplicate Ruby master - Bug #11284: String#upcase and String#downcase don't work for accented charactersRejectedActions
Actions

Also available in: Atom PDF

Like0
Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0Like0