Feature #22229: Allow GCI.escapeHTML to take a custom escape table

• Status: Open
• Assignee: -
• Target version: -

[byroot (Jean Boussier)]

Use case

CGI.escapeHTML has a fixed escape table:

    HTML_ESCAPE('\'', "'"),
    HTML_ESCAPE('&', "&"),
    HTML_ESCAPE('"', """),
    HTML_ESCAPE('<', "&lt;"),
    HTML_ESCAPE('>', "&gt;"),

But in some context you may want to escape more or less characters than that, or escape them differently.

One example of this is Active Support JSON serialization, which by defaults escape <>& as \u003e\u003c\u0026, to ensure that the generated
JSON can safely be interpolated inside a <script> tag without causing XSS vulnerabilities.

There's likely other use case as evidenced by the popularity of the htmlentities gem, which support several more escape tables.

Why not gsub ?

Today, this sort of escaping is performed using gsub, it works but is very noticeably slower than CGI.escapeHTML (see benchmarks on the PoC PR).
This is because the only way to use gsub with an escape table is to craft a regexp:

def escape(string, table)
  pattern = Regexp.union(table.keys)
  string.gsub(pattern, table)
end

puts escape("<script>", "<" => "&lt;", ">" => "&gt;")

If gsub could be used directly with the escape table it would certainly make it faster:

puts "<script>".gsub("<" => "&lt;", ">" => "&gt;")

But compared to CGI.escapeHTML it would still need to deal with noticeably more scenarios (multi character search, etc), so would always be slower than a dedicated escaping method.

CGI.escapeHTML can be more restrictive (single character search, etc).

Specification

GCI.escapeHTML("</script>", "<" => "&lt;", ">" => "&gt;")

• The escape table keys must be single characters (currently ASCII only, but could be muti-byte characters if deemed necessary).

Implementation

Pull Request: https://github.com/ruby/cgi/pull/55 (NB: it's at proof of concept / demo stage, if the feature is accepted I can polish it).

[byroot (Jean Boussier)]

• Description updated (diff)

[mdalessio (Mike Dalessio)]

I would also like to have this feature. Loofah (used by Rails's sanitization stack) currently has this code to re-implement escapeHTML with a custom table: https://github.com/flavorjones/loofah/blob/main/lib/loofah/html5/scrub.rb#L275-L299, as does the Rouge gem (and probably others).

[Eregon (Benoit Daloze)]

Using a Hash for this with escapeHTML* seems rather inefficient.
The ideal implementation should use a vectorized search, and that needs a more complex representation of the characters to search, which a Regexp is great at.

Regexp.union every time is of course very slow and a performance bug, but what about a fixed Regexp with the characters to escape?
That should be the right abstraction, and if that's slow I would say it's something to optimize in Regexp matching, because Regexp matching and gsub are designed very much for this use case.

[Eregon (Benoit Daloze)]

If there is any doubt that gsub can be faster, here is my blog post about it: https://eregon.me/blog/2025/03/14/matching-regexps-200-times-faster.html
Notably matching could be about 200 times faster.

[byroot (Jean Boussier)]

but what about a fixed Regexp with the characters to escape?

@Eregon (Benoit Daloze) please see the benchmark at https://github.com/ruby/cgi/pull/55, it doesn't compile a regexp every time and there's still a 5x difference.

I only did it in this in this example for simplicity, but also because you can't always have a static list upfront.

I would say it's something to optimize in Regexp matching,

I'm sure Regexp matching could be optimized further, but that's far from my area of expertise, and I made the point that by being limited to single characters, escapeHTML can be much easier to optimize much further.

[Eregon (Benoit Daloze)]

byroot (Jean Boussier) wrote in #note-5:

I made the point that by being limited to single characters, escapeHTML can be much easier to optimize much further.

That's something Regexp can optimize for and maybe already does to some extent, it can see which characters are in a character set and if they are all 1-byte, etc.
That's a great use case for SIMD, which Regexp should definitely use as it'd make all Regexps faster (if it doesn't already do that).

byroot (Jean Boussier) wrote in #note-5:

please see the benchmark at https://github.com/ruby/cgi/pull/55, it doesn't compile a regexp every time and there's still a 5x difference.

Interestingly, 2.83x when there is nothing to replace.
So from that it sounds a bit like gsub is quite a bit slower to replace than escapeHTML.
Yet it should be about the same since they both need to lookup the passed Hash when replacing.
That tells me there is something in gsub about replacing which is slower than it should be, independent of matching.

Looking at the PR diff it creates the escape table every time, that's the part that feels to me like this is not the right approach, because Regexp can build that escape table at compile time vs on every call.
gsub can't associate the given replacement with the escape table, but either way there are Hash lookups on every call if there are replacements, so that shouldn't matter much unless there are lots of replacements (gsub could also build a cache for that if there are many replacements, but might not matter much for perf).