Project

General

Profile

Feature #14618

Add display width method to String for CLI

Added by aycabta (aycabta .) about 1 month ago. Updated 30 days ago.

Status:
Open
Priority:
Normal
Assignee:
-
Target version:
-
[ruby-core:86213]

Description

Abstract

Unicode has display width data of characters, "Narrow" or "Wide".
For example, "A" is "Narrow", "💎" ("\u{1f48e}") is "Wide".
http://unicode.org/reports/tr11/
This data is very important for CLI tools.

Use-case

I'm developing Readline compatible library by pure Ruby implementation for Ruby core.
https://github.com/aycabta/reline

I'm discussing it with hsbt (Hiroshi SHIBATA), and I think that the pure Ruby version should be used only when the native extension version doesn't exist.
ref. https://bugs.ruby-lang.org/issues/11084
The Readline library is very important for that IRB always provides Readline's features.
So display width method is needed by Ruby core.

Implementation approach

Uses the official data table

Unicode Consortium provides display width data as "EastAsianWidth.txt".
http://www.unicode.org/Public/10.0.0/ucd/EastAsianWidth.txt

This name is based on historical reasons.
This table is not exclusively for East Asian's characters in the present day, for example, Emoji.

Uses new Regexp feature (work in progress)

I propose new Unicode properties for Onigmo like Perl's one.
https://github.com/k-takata/Onigmo/pull/102

I think that this is a better approach if the proposal for Onigmo is merged because String#grapheme_clusters what is based on Unicode specification uses Onigmo's feature inside.

Cases of other languages or libraries

Python: unicodedata.east_asian_width (standard library)
https://docs.python.org/3.6/library/unicodedata.html#unicodedata.east_asian_width

Perl: "East_Asian_Width: *" of Unicode properties (regular expression in language)
https://perldoc.perl.org/perluniprops.html

Go: golang.org/x/text/width
https://godoc.org/golang.org/x/text/width

PHP: mb_strwidth (standard library)
http://php.net/manual/en/function.mb-strwidth.php

JavaScript: eastasianwidth (npm library)
https://www.npmjs.com/package/eastasianwidth

RubyGems: unicode-display_width gem
https://rubygems.org/gems/unicode-display_width


Related issues

Related to Ruby trunk - Feature #13241: Method(s) to access Unicode properties for characters/stringsOpen

History

#1 [ruby-core:86206] Updated by Anonymous about 1 month ago

Dne 19.3.2018 v 20:00 aycabta@gmail.com napsal(a):

Use-case

I'm developing Readline compatible library by pure Ruby implementation for Ruby core.
https://github.com/aycabta/reline

I'm discussing it with hsbt (Hiroshi SHIBATA), and I think that the pure Ruby version should be used only when the native extension version doesn't exist.
ref. https://bugs.ruby-lang.org/issues/11084
The Readline library is very important for that IRB always provides Readline's features.

Just out of curiosity, why are you reimplementing readline in pure Ruby
if there exists rb-readline [1], used by Ruby Installer.

Vít

[1] https://github.com/ConnorAtherton/rb-readline

#2 [ruby-core:86209] Updated by shevegen (Robert A. Heiler) about 1 month ago

Martin Dürst is doing some unicode stuff; perhaps he can chime in
when he has some time.

#3 [ruby-core:86235] Updated by aycabta (aycabta .) 30 days ago

My e-mail to the mailing list [ruby-core] is not synced with Redmine.
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/86213

#4 Updated by duerst (Martin Dürst) 10 days ago

  • Related to Feature #13241: Method(s) to access Unicode properties for characters/strings added

Also available in: Atom PDF