Feature #19690: Add terminal colorizing gem to stdlib

• Status: Open
• Assignee: -
• Target version: -

[ccmywish (Aoran Zeng)]

Recently the new version of tracer gem caught my eye. But I've noticed an anti-DRY pattern.

I've noticed that many libs, CLI tools implement their own logic of color support on the terminal.

These libs/CLI tools don't explicitly require a gem from RubyGems.org may be because of these reasons:

1. It's very easy to directly monkey patch String class to support, such as bold, red, blue
2. The lib is very independent, the author doesn't want any dependency. See: https://github.com/ruby/tracer/blob/master/lib/tracer/color.rb

I hope the core team consider adding official support for terminal colorizing, because most Ruby developers live on the terminal. If we've added it, all gems/CLI tools developers can easily add simple styles. For example, adding color to gem and bundler command is attractive.

Next door community's npm seem much cooler by just using simple colors.

Could we coordinate with the gem rainbow's author to make it one of the stdlib or bundled gem like minitest?

[bkuhlmann (Brooke Kuhlmann)]

I'd like to see support for this in Ruby Core as well. To add to the above, I think it would be nice if there was a blend of good Object Oriented + Functional design with Rainbow as found in my Tone gem. The reason I bring this up is because testing terminal colors can be a pain so being able to not only encode colors, as the Rainbow gem, does but also decode colors, as the Tone gem does, would yield maximum benefit and a lot more joy to the community. The Tone gem also provides additional support for aliasing colors -- which would be nice to support too -- but might be out of scope for the core team to support (if considered). Anyway, some additional thoughts to the above, in case it's of help.

[st0012 (Stan Lo)]

Hi there. I think we're talking about two different needs here:

1. A coloring library for Ruby Core tooling projects, like IRB, tracer, and debug.
2. A coloring library for the community.

While they do have overlap, I think 1) has much fewer requirements in terms of functionalities and interface.

As the maintainer of IRB and the tracer gem, I agree that having 1) would be great. I'm not too sure if 2) needs to be fulfilled by an official library, but that's something you can ask more explicitly in a comment or another issue.

However, I don't think pulling the rainbow gem into Ruby Core would be the right solution for 1), as the mentioned tools don't need X11 colors, nor the color refinements, which are two major features of the library.

In my opinion, if we're going to have such a library, it will likely be just an extraction of (maybe some of) IRB's coloring logic.

[rubyFeedback (robert heiler)]

https://bugs.ruby-lang.org/issues/19690

Personally I support the original idea suggested by ccmywish so +1 from me
for this. I think it would be rather useful for potentially many ruby gems,
as well as many different ruby users out there, if we could add support
for colours "out of the box" on a fresh ruby installation, without
having to install additional gems.

Specifically this reasoning I agree with mostly:

I hope the core team consider adding official support for terminal
colorizing, because most Ruby developers live on the terminal.

I may not use the same argument as ccmywish does (I am not sure if
"most Ruby developers live on the terminal", for instance) - but I agree
that there are many ruby "scripts" out there that make use of colours.

For instance, in my own gem that supports colours (yes, I also created
one because I really need colours and find them to be very helpful), one
test-method outputs something like this on the commandline:

https://i.imgur.com/F6kac8W.png

That gem has almost 100.000 downloads too \o/ so while many of these downloads
come from bots/scripts automatically, there are also quite a few human
users (or, at the least, they tried it out to see whether it is useful
to them).

There are several more popular gems for colour-support, on rubygems.org.

For instance, the "color" gem has about 10 million downloads; the
"tty-color" gem has almost 40 million downloads; red-colors (used/developed
by Kenta Murata and Sutou Kouhei) has half a million downloads - and
various more gems; there is also term-ansicolors, IRB colours, perhaps
bundler too (I think it uses colours, such as in thor/
and in lib/bundler/ui/shell.rb @shell.set_color(string, *color),
among other places). Some of that functionality is probably overlapping.

So I believe we can conclude that colour-related support definitely may
be useful for ruby users, thus reinforcing ccmywish's primary suggestion.

ccmywish wrote:

Next door community's npm seem much cooler by just using simple colors.

Can we describe how npm makes colours "cooler" or more useful? I ask
this so that we can focus on the "good bits", and then eventually integrate
them into ruby. I genuinely don't know; aside from glancing at a few
tutorials I never really used npm.

I agree that colours are very helpful on the commandline; I combine them a
lot with Emojis. I saw this on first OSX some years ago, I think, where
emojis are used such as for a todo list or to indicate some important
parts. It's a bit like a "mini-GUI", just a TUI (terminal user interface).

We should keep track of what such a colour-support gem in ruby should
support - the main use cases. And document them - kind of like a
specification.

bkuhlmann wrote:

The reason I bring this up is because testing terminal colors
can be a pain so being able to not only encode colors, as the
Rainbow gem, does but also decode colors

We could add the test-code to that new gem too, and give "downstream"
users a way to test for things and functionality. I have little to
no experience in regards to testing terminal colours, but it strikes
me as a solid argument that there is a need to test for terminal
colour support - different terminals may require different support
here; I remember the XTERM variable or so, where one is assumed
to set what the terminal supports. (Personally I use mostly KDE
konsole these days and via RGB support I seem to "have all the
colours in the world", but I guess it may depend on the terminal
and system at hand; perhaps some older systems don't have the
same support, perhaps embedded systems either, so I concur that
this could be useful to have by default.)

st0012 wrote:

I think we're talking about two different needs here:

A coloring library for Ruby Core tooling projects, like IRB, tracer, and debug.
A coloring library for the community.

I do not think we have two different needs - or better, use cases - here.

I feel that the second point you mention is very similar to the first point
and vice versa.

Specifically I am not convinced why we should arbitrarily distinguish between
"colours support in IRB is good (because of debugging reasons or a similar
rationale), but for regular commandline scripts it is super super bad" all
of a sudden. I don't see it that way.

I see colours potentially as always useful and helpful; of course in my own
gems I give the option to disable colour output (usually via --disable-colours
/ --disable-colors). That way people can decide whether they want colours.
As I use my own gems too, the default is to make use of colours. Colours
help me a LOT. For instance, warnings/errors I tend to display in a red
colour (actually the HTML colour tomato, as I find it more elegant
than default red on a black background).

So I don't think these are truly separate use cases. To me the overlap is
more or less identical. I am not saying your opinion is incorrect, of
course - it's just different. From my point of view I don't see that
as a truly separate aspect.

st0012 wrote:

I think 1) has much fewer requirements in terms of functionalities
and interface.

So you appear to reason in regards to amount of code / how complete the
API should be. I am fine with that as a design consideration; I just
disagree that the reasoning should be about arbitrary distinctions
between "tooling" use cases and "community needs" having different
needs. Ultimately the question is what colours should be used for. I
think the primary use case is that most people will find colours
useful; evidently this is not always the case, e. g. red-green
colour blindness or full colour blindness, but that is probably
a smaller domain of users - and for users who do not want colours
we should always support the option to not make use of colours. But
the remaining part of folks may probably appreciate colours.

Some of this may be up to one's personal preferences, e. g. just
a few colours, versus rainbow-colour streaks where the design
consideration is more on fancy effects than on highlighting the
important parts of sentences/text.

How that API should actually be can be debated. I have no really strong
opinions here. We should, however had, specify this as well as the use
case(s). To cover these use cases.

Personally I am a bit biased here in that I think the colour-related
aspects should be very flexible and extensive, highly documented
and come with several examples. So my opinion is more towards the
"batteries included". I am fine with other preferences though - my
thumbs up goes mostly towards the idea of having a colour-related
gem/code in ruby by default, when someone uses ruby.

For instance, I do not really use "hello world".red (I don't want to
modify class String like that, not even via refinements), but I remember
flori (flgr) had this already back in 2006 or so, I think (see his
gem https://github.com/flori/term-ansicolor ), and I am fine if that
should be available as an option too. Even though I don't need it.
People should be able to pick/select what they want. That colours-related
gem should support such use cases, without them getting "leaky" (leaky
duck patching, also known as monkey patching, may not always be great,
even though it allows for flexibility).

My argument here is that if we have a very flexible colours gem
that includes perhaps 80% or 90% of the most common use cases out
there, and allows us to be modular and flexible, then this is perfectly
fine to have "out of the box", e. g. already when someone installs
ruby on a new machine, or has it available on a fedora/centos/debian
install, by default. This would be nice to have, and it can be
maintained similar to e. g. Hiroshi's example of reline replacing
the functionality of readline eventually (pure ruby alternative) -
see the project on github. (I considered asking about reline,
if it covers all use cases of readline and this is documented,
but I think once reline will be used by more people we'll get
more reports "in", in this regard anyway.)

I should also say that I have no real preference in how extensive
that colours-related gem should be. That is, we could create a new
gem that unifies all that aspects; or we could re-use another
already existing gem. It's all fine by me personally; I have no
strong opinion here either way. The only strong opinion I have
is that it:

• should work
• be extensively documented
• be useful, and
• come with examples (and we can integrate the goal of having a
  test/spec suite people can test for and re-use too, as API as
  well; in a few projects I need to "uncolourize" a String too,
  so this could be another use case, depending on whether other
  people ever run into such a use case).

st0012 wrote:

However, I don't think pulling the rainbow gem into Ruby Core
would be the right solution for 1), as the mentioned tools
don't need X11 colors, nor the color refinements, which
are two major features of the library.

I don't have a strong opinion in how big the API should be,
even though personally I lean more towards a "complete package"
in regards to colours. That way people could focus on it and
re-use, so some of these colours-related gems may no longer
be necessary. I see this as a good thing, but it depends on
one's point of view. What we may all have to agree is that
there are quite a few colours-related gems on rubygems, and
some of them are quite popular as well.

I agree that X11 is somewhat specific to Linux/UNIX, but there
are also HTML colours, such as steelblue, lightblue, as well
as RGB values (KDE konsole allows us to use RGB values, and
this is super-nice). I lean more towards the "make that gem
useful by default". See wikipedia:

https://en.wikipedia.org/wiki/X11_color_names

I don't see it quite as strict as you may see it; for instance,
"Dark Turquoise" is "#00CED1" is RGB: 0% 81% 82%. To me this
is all ultimately the same, but for a user the string name
"dark turquoise" may be easier to remember, which is why I
actually like HTML colours. But either way I think it is all
the same really. (That gem should also support colour-conversions
by the way; this was one reason I created my own gem too,
because I could not remember the RGB values or hexadecimal
values, yet needed this sometimes e. g. KDE konsole wants
the RGB values.)

Perhaps we can split this issue here a bit, that is, first, whether
people think this may be a good idea or not, to have colours support
in ruby by deafult - we could ask this to the ruby core team.

Then, if matz / the ruby core team gives thumbs up, I think it would
help motivate people to eventually integrate that into ruby in one
way or another. Let's perhaps aim for this for the xmas release or
so, just to randomly give a time frame. Or next year - it is not
that important how long it takes, as long as we can clearly see
whether this may be useful to have or not. I think it would be
useful to have.

Then, when we know the ruby core team thinks this idea has merit,
we can discuss / vote how much functionality should be included
into it actually. That way we don't depend on already existing
gems and instead can focus on "which use cases are quite
popular or useful". Some of it may be based on personal preference,
but we can specify and document how extensive that gem should be.
Once that is decided, the rest should be quite simple and
straightforward - I am sure some of the code is overlapping
and already existing so that could be used as-is.

Perhaps something like a poll could be done in regards to how extensive
the API should be, after some discussion what could be useful to have.

But as I said, I am really fine if people have a different opinion here -
I think this is a secondary aspect anyway, we should first focus on the
colours-support by-default (yes/no), and if it is yes we can highlight
what should then be part of this. See my comment above about designing
towards a gem that should cover xyz documented use cases. Having
this documented is important, in my opinion, because new users could
then just use that gem, and have colours-support for their own ruby
"scripts". Otherwise they may have to search the world wide web or
ask questions, and then people may point at already existing gems
that would then seem "so much better" than what ruby may make available
as stdlib gem. That is why I think the documentation should be
super-stellar and of high quality: make people WANT to use that
stdlib gem (if approved and agreed that this is a good idea to have).

st0012 wrote:

In my opinion, if we're going to have such a library, it will
likely be just an extraction of (maybe some of) IRB's coloring logic.

I am not sure. I also don't know IRB's colouring logic either so I
have no idea if it covers all use cases; I think recall having had
issues on windows, but this may be in regards to tab-completion or
something like that, where IRB seems to be doing something that
takes a second or two. I am more agnostic here; we should go about
this step-wise. Perhaps as bullet points what features should all
be part of it. Note that referring to IRB's coloring logic has
a similar problem as referring to rainbow's coloring logic - we
all focus on existing projects. ;)

IMO it may be better to see this more as a grounds-up bottom-up
design; once we cover that we could combine what seems useful
for many ruby users out there.

Does someone have an idea how to determine the "most frequent use
cases"? I only know about my own use cases really, and a few more
colour related output on the commandline, such as (I think) for bundler
and rubocop and rails (these yield some colour support on the
commandline if I recall correctly; I am not too familiar with
the tty-related gems, so perhaps people who use the tty-related
gems can talk about that, as well as how colours are used via
the tty/TUI family of gems). These seem to be fairly frequent
use cases; perhaps also colour conversion, but I don't know how
popular these are. Does anyone have some kind of script to
determine how popular colours-support may be in different
scripts and projects? We probably will not be able to estimate
all the code out there in ruby using colours, but we can
probably get some idea about the "top 5" or "top 10" use cases
if we have some script that can be run to determine that. This
could then help decide how the API should be. I think not
everyone knows all these different gems; for instance, I have
no idea "tty-color" does, but it is definitely the most
popular one out there.

[rubyFeedback (robert heiler)]

One more smaller thing to add: I don't know how colours are
used on embedded / constraint system, so anyone with some
experience with mruby and embedded system could perhaps also
comment on this. Someone using raspberry or arduino, whether
they use colours on these systems, and how popular that is
there.

[hsbt (Hiroshi SHIBATA)]

• Related to Feature #20100: [Feature] Better / easier support for (optional) colours, on the commandline, in case a parsing error happened? added