How to deal with Trojan Source vulnerability
The "Torjan Source" vulnerability recently has caught some attention.
The vulnerability involves using certain combinations of Unicode characters to let source code look like it is correct (and therefore pass code review,...) but actually do something else than intended.
For background, please see discussion on KrebsonSecurity (https://krebsonsecurity.com/2021/11/trojan-source-bug-threatens-the-security-of-all-code/) and the Web site (https://www.trojansource.codes/) and original paper (https://www.trojansource.codes/trojan-source.pdf).
I contacted the Ruby security list, which was already aware of the issue, and we agreed to discuss this here because the vulnerability is already public.
The paper focuses on the use of [A] Directional Formatting Characters (*1) in string constants, comments, and similar constructs to change the visual appearance of code outside these constructs. There are related vulnerabilities, namely the use of [B] non-spacing (and therefore mostly invisible) characters e.g. in variable names, and the use of [C] mixed-script identifiers, which also lets some variable names look identical even if they are not.
Some languages, such as Rust, have addressed [A] (see https://blog.rust-lang.org/2021/11/01/cve-2021-42574.html) by requiring escapes to be used for the relevant characters in source. On the other hand, people such as Russ Cox think compilers are the wrong place to address the issue; it should be addressed in editors and similar tools (see https://research.swtch.com/trojan). Github now warns about
The question is what Ruby should do, if anything.
Addressing [A] similar to how Rust does it can be done relatively easily. If that's done, I'd prefer to only reject incomplete Bidi control sequences, which is a bit more complicated. In particular, string interpolation needs a very careful analysis.
For [B], I'll open a separate issue.
For [C], we have all data about scripts, but the way it's currently structured makes finding out which character a script belongs to quite inefficient.
(*1) "Directional Formatting Character" is the official Unicode term (see https://www.unicode.org/reports/tr9/#Directional_Formatting_Characters). The terms "Bidi/Bidirectional control" or "Bidi/Bidirectional control character" are also used. Overall, there are 9 such characters. Unfortunately, both the paper and KrebsonSecurity use the term "Bidi Override", which is highly misleading. The term “Bidi Override” is reserved for two characters only:
LRO, U+202D, Left-to-Right Override, and RLO, U+202E, Right-to-Left Override (see Table 1 in the paper). It is also used for the phenomenon associated with these two characters, a “hard” override (i.e. affecting all characters including e.g. the Latin alphabet), and mechanisms in other technology that achieve the same (e.g. the HTML bdo element (https://html.spec.whatwg.org/#the-bdo-element) or the ‘bidi-override’ value of the unicode-bidi property in CSS (https://www.w3.org/TR/CSS2/visuren.html#propdef-unicode-bidi)).