Feature #18996: Proposal: Introduce new APIs to reline for changing dialog UI colours

• Status: Closed
• Assignee: -
• Target version: -

[st0012 (Stan Lo)]

TL;DR

I want to add APIs to reline for changing its dialog item's colors.
The APIs I want to add actually have been merged but becaue:

1. This is a design change
2. The maintainer @aycabta (aycabta .) is not available to approve nor reject them

I want to raise it here to decide if we should:

1. Drop them
2. Modify them
3. Officiallty accept them

Background

After version 1.4, irb provides autocompletion support, which is a great feature and has increased many developers' productivity significantly.
But there's an user-experience issue: the completion items' UI colors (set in reline) are not configurable. So depending on the user's terminal theme, some may find it hard to use because the background and the text having low-contrast colors, like this:

And if that happens, the user has no way to fix it. This caused users to open issues like:

• https://github.com/ruby/irb/issues/351
• https://github.com/ruby/irb/issues/328

for being able to change it.

Some users even decided to disable it completely because the colors are unreadable to them. I have also seen people sharingtips for disabling this feature: example. So I believe it may be bothering many developers.

Personally I really like this feature but the background also bothers me:

And that's why I want to improve it by making the colors configurable and potentially also by providing simple light/dark themes from irb.

Proposal

For the dialog UI, there are 2 element states: highlighted and default. In irb's case, the selected completion candidate will be highlighted, and the rest of options will be default. And each state has 2 colors: foreground (text) and background (block).

This means the reline should allow irb and/or users to configure:

• Default items' foreground color
• Default items' background color
• Highlighted items' foreground color
• Highlighted items' background color

That brings us to these APIs:

• Reline.dialog_default_fg_color
• Reline.dialog_default_bg_color
• Reline.dialog_highlight_fg_color
• Reline.dialog_highlight_bg_color

And because reline only supports coloring through ANSI sequences, these APIs only has 8 available colors if we exclude their bright variants:

• Black
• Red
• Green
• Yellow
• Blue
• Magenta
• Cyan
• White

Given the limited options and also to prevent users from entering non-color ANSI sequences, these APIs only take color names directly:

• :black
• :red
• :green
• :yellow
• :blue
• :magenta
• :cyan
• :white

Example:

Reline.dialog_default_bg_color = :black
puts Reline.dialog_default_bg_color_sequence #=> 40
Reline.dialog_default_fg_color = :white
puts Reline.dialog_default_fg_color_sequence #=> 37
Reline.dialog_highlight_bg_color = :blue
puts Reline.dialog_highlight_bg_color_sequence #=> 34
Reline.dialog_highlight_fg_color = :black
puts Reline.dialog_highlight_fg_color_sequence #=> 30

I have made a proof of concept PR on irb to show what these APIs can achieve if they or similar ones are adopted.

Related PRs

The related changes are made through multiple PRs:

• Initial APIs PR by @pocari
• PR to improve the APIs and make them safer to use
• PR to rename the APIs

Other Thoughts

This is more of a concern on the irb part, but to make the UI looks comfortable, I think it's better to follow these conditions:

1. An item's foreground and background colors should have high contrast with each other so the texts (foreground) are readable.
2. For the highlighted item, its background color should be easily distinguishable from the rest of default items.
3. When using dark terminal themes, the default items' background is better to be dark as well.

[k0kubun (Takashi Kokubun)]

Some users even decided to disable it completely because the colors are unreadable to them. I have also seen people sharing tips for disabling this feature: example.

JFYI, I'm one of the people who are disabling IRB completion, but it was not because I can't configure colors but because the prompt position moves a lot depending on the size of the completion window. If we are interested in addressing the problem you described, we should check if the default color is the primary reason for those people to disable it. It's unclear from the example tweet alone whether the author was annoyed by the color or not.

That brings us to these APIs

I agree with making those four things configurable. Supporting only ANSI escape sequences for now makes sense too. I don't have a strong preference on Reline-level APIs, but these APIs look okay.

I have made a proof of concept PR on irb to show what these APIs can achieve if they or similar ones are adopted.

Unlike Reline's API, I have a few thoughts on how IRB should be configured, but it's not the scope of this ticket. So I'll discuss it elsewhere :)

[st0012 (Stan Lo)]

k0kubun (Takashi Kokubun) wrote in #note-1:

JFYI, I'm one of the people who are disabling IRB completion, but it was not because I can't configure colors but because the prompt position moves a lot depending on the size of the completion window. If we are interested in addressing the problem you described, we should check if the default color is the primary reason for those people to disable it. It's unclear from the example tweet alone whether the author was annoyed by the color or not.

Based on the screenshot of that tweet, I believe the coloring issue is implied.
But even if we excluded that one, coloring is also mentioned in the issues I included. Here are some quotes from their comments:

The autocomplete list is shown in color and cannot be read at all if the environment and desktop colors are set in a different way

And the autocomplete is a bright cyan bg color that is completely unreadable.

When starting IRB in Ruby 3.1, autocomplete suggestions are shown but hard to read due to the default background color. Ideally, the default background color settings would ensure enough contrast to make the text legible.

And here are other tweets complaining about irb coloring:

https://twitter.com/mtabetz/status/1571378786700087297 (posted just a few hours before this comment)
https://twitter.com/ryanmammina/status/1474764022524297219

I'm not saying it's the only reason people disable the autocompletion feature. I also heard people disabling it because it slows SSH connection down for example.

But from my personal experience (as mentioned with a screenshot earlier) and the reactions I've seen on Twitter or Ruby community spaces (private Slack...etc.), background color is definitely one thing we can and should improve.

[mame (Yusuke Endoh)]

Discussed at the dev meeting. For the API style, how about Reline.color_config with a hash object like this?

Reline.color_config.merge!({
  dialog_default_bg_color: :white,
  dialog_default_fg_color: :black,
  dialog_highlight_bg_color: :white,
  dialog_highlight_fg_color: :black,
})

Note that we may want to add settings in future (e.g., more attributes like italic, bold, etc). We will need to add more methods for each additional setting if we choose a style of Reline.dialog_default_bg_color = :white.

Also, how did you choose the names "default", "highlight", "bg_color", etc.? It would be good to clarify the rationale.
For example, Terminal.app uses the term "selection" instead of "highlight".
The article of Wikipedia "ANSI escape code" says "Set background color" instead of "bg_color". (It would be better to find the original source of ANSI escape code instead of the Wikipedia article.)

[st0012 (Stan Lo)]

It would be better to find the original source of ANSI escape code instead of the Wikipedia article.

The 8-color part of ANSI escape code (not the entire standard) were defined in the ECMA-48 standard 's page 62. (This standard later became ISO-6429)

In the standard, it uses [color] display to refer to foreground colors, and [color] background for background colors. However, it's worth noting that in other terminal related documents, like the wiki page I linked above or this Console Virtual Terminal Sequences manual from Microsoft, foreground and background are the more common terms.

So I think we should stick with foreground and background as display is more ambiguous.

bg_color or background_color

I'm fine with both the longer foreground_color and background_color or the short versions like fg_color and bg_color.

New Configuration Interface

After reading the feedback and doing more research, I want to propose a new configuration interface:

Reline.dialog_config.merge!({
  background_color: :black,
  foreground_color: :white,
  highlighted_background_color: :white,
  highlighted_foreground_color: :black,

  # and in the future, if we want to introduce more styles
  text_style: [:italic, :underscore],
  highlighted_text_style: [:bold],
})

Why dialog_config instead of color_config

When it comes to UI configurations, I think it makes more sense to group the options by the component (dialog) than the type of property (color). For example, when defining CSS we declare a selector's styles (background, font...etc.) in the same block.

.dropdown-content {  
  position: absolute;  
  background-color: #f9f9f9;  
  min-width: 160px;  
  box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);  
  padding: 12px 16px;  
  z-index: 1;
}

This also means that we don't need to have the component prefix (dialog_) in every option.

Item states: highlighted vs selection

reline's' dialog can have 2 types of usages:

1. Dropdown selection, like irb's autocompletion
2. Content display, like irb's documentation dialog

The difference is not well documented but the behavior is controlled by the dialog's pointer attribute. If it's assigned to an integer that matches one of the items' index, the item will be highlighted (scenario 1). If it's assigned with a nil, then all contents will be displayed the same way (scenario 2).

Given that reline's dialog isn't just for autocompletion dropdown, I think using selection as a state name is too narrow. And purpose of pointer is to display a specific content with different set of color(s), so I think highlighted is a better name here.

The Current (v0.3.1) Implementation's bg_color

I forgot to mention that in reline v0.3.1, there's already a way to configure a dialog's background color by passing bg_color to DialogRenderInfo. And irb is already using it to set its documentation dialog's background color.

However, I think it's not the API we should settle with. Let me explain why:

To build a reline dialog, there are 2 levels of APIs to use. The lowest level is to build a proc that handles the dialog logic (which needs to returns a DialogRenderInfo), and use the add_dialog_proc method to register it. This is what irb's documentation dialog uses (the complete SHOW_DOC_DIALOG proc). And the bg_color attribute is only accessible at this level.

And on top of that, reline provides a default dialog proc for autocompletion (DEFAULT_DIALOG_PROC_AUTOCOMPLETE). In this case, the user only needs to generate content candidates in a proc, and register it with Reline.completion_proc (example from irb). And since this approach doesn't build a DialogRenderInfo from ground up, it won't be able to configure bg_color. So even if we add a fg_color to the DialogRenderInfo struct, we still won't be able to configure it from irb.

Therefore, I think the proposed configuration interface is still necessary.

[k0kubun (Takashi Kokubun)]

Naming Convention

Looking at #note-3, the last dev-meeting's primary ask was to provide a rationale for naming the configurations. So far, the following things have been mentioned:

• ECMA-48 standard
• ANSI escape code - Wikipedia
• Console Virtual Terminal Sequences - Windows Terminal
• Terminal.app

It seems like #note-4 discussed individual configurations with different reasoning, however, because the concern is about additional future settings, we should choose a consistent way to name them if possible.

My two cents: Given that these configurations are tightly coupled with terminal applications, I think it should be consistent with the names used in terminal applications, at least for concepts that exist in terminal apps, e.g. foreground, background. For concepts that don't exist in terminal apps, we could use what Reline names, e.g. dialog, or carefully pick an arbitrary name that makes the most sense as a last resort.

AutoCompletion vs Documentation

[Feature #19010] suggested the possibility that, if we were to introduce a color theme to IRB, we'd like to configure colors of other things such as syntax highlight as well. When a color theme can specify different colors for different components, we might want to use different colors for auto-completion and documentation in the future.

If auto-completion and documentation are implemented differently by SHOW_DOC_DIALOG and DEFAULT_DIALOG_PROC_AUTOCOMPLETE, why don't you just separate color configs for those two things? If you introduce only a color config for auto-completion, which is the actual scope of the problem discussed in this ticket in the first place, for the place of highlighted, you'll be able to comfortably use the name that is consistent with terminal apps: selection.

[st0012 (Stan Lo)]

• File Screenshot 2022-10-18 at 15.25.36.png Screenshot 2022-10-18 at 15.25.36.png added

If we want to establish naming convention based on terminal apps, I think I'd pick iTerm2 as the reference because 1) its very popular and 2) it allows more customisation than Terminal.app:

And if we use its naming convention for the 4 colours, they'll be:

• Default items' foreground color: foreground_color
• Default items' background color: background_color
• Highlighted items' foreground color: selected_text_color
• Highlighted items' background color: selection_color

If auto-completion and documentation are implemented differently by SHOW_DOC_DIALOG and DEFAULT_DIALOG_PROC_AUTOCOMPLETE, why don't you just separate color configs for those two things

That's a good point. I guess we can just expose colour options at DialogRenderInfo level, like

DialogRenderInfo.new(
  pos: Reline::CursorPos.new(x, y), 
  contents: contents, 
  width: width, 
  foreground_color: :white,
  background_color: :black,
  selected_text_color: :black,
  selection_color: :white
)

Then we can avoid having a top-level config that may be hard to name in the future.

But then we also need to think of a way to pass them to DEFAULT_DIALOG_PROC_AUTOCOMPLETE because it locates inside reline and it's not patchable.

Given that reline already exposes several completion-specific configs, like

Reline.completion_append_character = nil
Reline.completer_quote_characters = ''
Reline.completion_proc = IRB::InputCompletor::CompletionProc

Do you think it'd make sense to have one for the colours too? For example:

Reline.completer_colors = {
  foreground_color: :white,
  background_color: :black,
  selected_text_color: :black,
  selection_color: :white
}

[k0kubun (Takashi Kokubun)]

And if we use its naming convention for the 4 colours, they'll be:
Default items' foreground color: foreground_color
Default items' background color: background_color
Highlighted items' foreground color: selected_text_color
Highlighted items' background color: selection_color

Sounds good. I think it'd be also fair to make the APIs a bit more consistent by using selection_foreground_color and selection_background_color, but then you'll lose consistency with iTerm2 a little and the names end up being longer.

Do you think it'd make sense to have one for the colours too?

Yes, it seems easier to configure than DialogRenderInfo. As to completion vs completer, git grep in the IRB repository seems to suggest that "completion" is used more commonly throughout the repository, so I'd use completion_colors. At least completier seems like a typo of completer.

[st0012 (Stan Lo)]

Do you think it'd make sense to have one for the colours too?

Yes, it seems easier to configure than DialogRenderInfo. As to completion vs completer, git grep in the IRB repository seems to suggest that "completion" is used more commonly throughout the repository, so I'd use completion_colors. At least completier seems like a typo of completer.

Yes completier was a typo. I've corrected it.

I did a quick search and it looks like completer_* methods, completer_word_break_characters and completer_quote_characters, were added for compatibility with Readline. So I agree that we should go with completion_colors instead.

To summarize the current consensus:

1. The 4 colour options will be named as the following to match iTerm2's naming convention:

   

   • Default items' foreground color: foreground_color
   • Default items' background color: background_color
   • Highlighted items' foreground color: selected_text_color
   • Highlighted items' background color: selection_color

2. The colour options will be added to

   1. The DialogRenderInfo level

      DialogRenderInfo.new(
        pos: Reline::CursorPos.new(x, y), 
        contents: contents, 
        width: width, 
        foreground_color: :white,
        background_color: :black,
        selected_text_color: :black,
        selection_color: :white
      )
      

   2. And a new completion-specific API

      Reline.completion_colors = {
        foreground_color: :white,
        background_color: :black,
        selected_text_color: :black,
        selection_color: :white
      }
      

[k0kubun (Takashi Kokubun)]

Nice summary. This seems like a solid proposal.

[hsbt (Hiroshi SHIBATA)]

I'm +1 to https://bugs.ruby-lang.org/issues/18996#note-8 basically.

And a new completion-specific API

Reline.completion_colors = {
  foreground_color: :white,
  background_color: :black,
  selected_text_color: :black,
  selection_color: :white
}

...colors = { ..._color: ...} is a little of redundant. Can we use like this?

Reline.completion_colors = {
  foreground: :white,
  background: :black,
  selected_text: :black,
  selection: :white
}

Note: This suggestion is not strong opinion.

[st0012 (Stan Lo)]

hsbt (Hiroshi SHIBATA) wrote in #note-10:

Reline.completion_colors = {
  foreground: :white,
  background: :black,
  selected_text: :black,
  selection: :white
}

I agree that color postfix is a bit redundant. But I still lean slightly toward having it because it makes passing values easier:

DialogRenderInfo.new(
  pos: Reline::CursorPos.new(x, y), 
  contents: contents, 
  width: width, 
  **Reline.completion_colors
)

And when doing so, users get errors when they have typos or unsupported attributes in their Reline.completion_colors, for example.

But I'll take either one decided in the dev meeting.

[mame (Yusuke Endoh)]

• Related to Feature #19010: Follow up of #18996: Support changing irb's autocompletion background added

[hasumikin (hitoshi hasumi)]

I opened a PR to restart this discussion:
https://github.com/ruby/reline/pull/552

[st0012 (Stan Lo)]

@hasumikin (hitoshi hasumi) proposed a more powerful and mature design and it has been merged. So I think both this and #19010 can be closed :-)

[hsbt (Hiroshi SHIBATA)]

• Status changed from Open to Closed