Feature #13839: String Interpolation Statements

• Status: Rejected
• Assignee: -
• Target version: -

Hello!

Here is a KISS implementation of a template engine in Ruby:

class Template

  attr_reader :input

  def initialize(input)
    @input = input
  end

  def output
    "output = %\0" + @input.gsub("{%", "\0\n").gsub("%}", "\noutput += %\0") + "\0"
  end

  def render(binding)
    eval(output, binding)
  end

end

Usage:

{% if true %}
  Hello #{'World'}
{% end %}

Template.new('...').render(binding)

It's kind of a hack on top of Ruby string interpolation, so it's hell fast (~4 times faster than ERB).

Could it be a good idea to implement this kind of statements directly in Ruby string interpolation? Maybe a syntax like that:

"%{3.times do}Hello #{'World'}%{end}"

So Ruby would have a fast minimal native template engine, with #{expressions} and %{statements}:

eval(File.read("..."), binding)

[jeremyevans0 (Jeremy Evans)]

It might be better to compare this to Erubi, the current default ERB template processor in Rails and Tilt. While this approach is fast for small strings, it's actually slower for large strings (probably due to the use of += instead of <<). Additionally, in most cases when you are using templates, you want to cache the resulting ruby code so that repeated rendering does not need to reparse the template input. When you start caching the template object, the performance advantage disappears for small strings, and the performance disadvantage becomes larger for large strings.

I don't think ruby should include this this in core or stdlib. I think it would be best to have it as an gem, possibly integrating with Tilt. But I'll admit I'm biased in this respect, as I'm the maintainer of Erubi.

Here's example benchmark code:

class Template

  attr_reader :input

  def initialize(input)
    @input = input
  end

  def output
    "output = %\0" + @input.gsub("{%", "\0\n").gsub("%}", "\noutput += %\0") + "\0"
  end

  def render(binding)
    eval(output, binding)
  end

end

require 'erubi'

class ErubiTemplate
  attr_reader :input

  def initialize(input)
    @input = input
    @src = Erubi::Engine.new(input).src
  end

  def render(binding)
    eval(@src, binding)
  end
end

template_input = <<'END'
{% if true %}
  Hello #{"World"}
{% end %}
END

erubi_input = <<'END'
<% if true %>
  Hello <%= "World" %>
<% end %>
END

require 'benchmark/ips'

Benchmark.ips do |x|
  x.report("Template-small"){Template.new(template_input).render(binding)}
  x.report("Erubi-small"){ErubiTemplate.new(erubi_input).render(binding)}
  x.compare!
end

template = Template.new(template_input)
erubi = ErubiTemplate.new(erubi_input)

Benchmark.ips do |x|
  x.report("Template-small-cache"){template.render(binding)}
  x.report("Erubi-small-cache"){erubi.render(binding)}
  x.compare!
end

template_input *= 10000
erubi_input *= 10000

Benchmark.ips do |x|
  x.report("Template-large"){Template.new(template_input).render(binding)}
  x.report("Erubi-large"){ErubiTemplate.new(erubi_input).render(binding)}
  x.compare!
end

template = Template.new(template_input)
erubi = ErubiTemplate.new(erubi_input)

Benchmark.ips do |x|
  x.report("Template-large-cache"){template.render(binding)}
  x.report("Erubi-large-cache"){erubi.render(binding)}
  x.compare!
end

and results (slightly reformatted for easier viewing):

Calculating -------------------------------------
      Template-small     19.271k (_ 0.7%) i/s -     97.515k in   5.060508s
         Erubi-small      9.123k (_ 0.4%) i/s -     46.163k in   5.060080s

Comparison:
      Template-small:    19270.8 i/s
         Erubi-small:     9123.2 i/s - 2.11x  slower


Calculating -------------------------------------
Template-small-cache     19.962k (_ 0.5%) i/s -    100.980k in   5.058694s
   Erubi-small-cache     21.568k (_ 0.5%) i/s -    108.900k in   5.049226s

Comparison:
   Erubi-small-cache:    21568.2 i/s
Template-small-cache:    19962.1 i/s - 1.08x  slower


Calculating -------------------------------------
      Template-large      0.451  (_ 0.0%) i/s -      3.000  in   6.658039s
         Erubi-large      0.693  (_ 0.0%) i/s -      4.000  in   5.771604s

Comparison:
         Erubi-large:        0.7 i/s
      Template-large:        0.5 i/s - 1.54x  slower


Calculating -------------------------------------
Template-large-cache      0.449  (_ 0.0%) i/s -      3.000  in   6.682826s
   Erubi-large-cache      3.366  (_ 0.0%) i/s -     17.000  in   5.062812s

Comparison:
   Erubi-large-cache:        3.4 i/s
Template-large-cache:        0.4 i/s - 7.50x  slower

Thanks for your feedback Jeremy. :) (I have much gratitude for all your work in Ruby!)

Yes, I didn't think of this as a replacement for a full featured template engine (with cache, helpers and so), but more as a quick and powerful way to embed Ruby in various documents.

I was listening a talk by Rasmus Lerdorf, the creator of PHP. He said that one of the most appealing feature he added to PHP was to include trivial templating out of the box. And it's true, in Ruby or Python, it's harder to get started, you have to require an external library and you are overwhelmed by all these competing choices: Haml ? ERB (which variant?)

But I don't know really how you guys feel about that, I am not a language designer. :)

By the way Nao submitted a patch for improving string interpolation performance:
https://bugs.ruby-lang.org/issues/13587

Good idea on replacing += with <<, it really improved the code speed for large strings:

Calculating -------------------------------------
      Template-small     91.792k (± 0.9%) i/s -    464.308k in   5.058681s
         Erubi-small     39.139k (± 2.4%) i/s -    197.166k in   5.040678s

Comparison:
      Template-small:    91792.2 i/s
         Erubi-small:    39139.1 i/s - 2.35x  slower


Calculating -------------------------------------
Template-small-cache     94.974k (± 1.6%) i/s -    476.632k in   5.019886s
   Erubi-small-cache     91.008k (± 0.7%) i/s -    460.512k in   5.060385s

Comparison:
Template-small-cache:    94973.6 i/s
   Erubi-small-cache:    91008.5 i/s - 1.04x  slower


Calculating -------------------------------------
      Template-large     19.171  (± 5.2%) i/s -     96.000  in   5.023298s
         Erubi-large      2.366  (± 0.0%) i/s -     12.000  in   5.075102s

Comparison:
      Template-large:       19.2 i/s
         Erubi-large:        2.4 i/s - 8.10x  slower


Calculating -------------------------------------
Template-large-cache     19.560  (±10.2%) i/s -     97.000  in   5.014184s
   Erubi-large-cache     16.434  (±12.2%) i/s -     81.000  in   5.040727s

Comparison:
Template-large-cache:       19.6 i/s
   Erubi-large-cache:       16.4 i/s - same-ish: difference falls within error

[k0kubun (Takashi Kokubun)]

• Status changed from Open to Feedback

You wrote:

{% if true %}
  Hello #{'World'}
{% end %}

Template.new('...').render(binding)

You should write expected code in valid Ruby syntax. Guessing from "%{3.times do}Hello #{'World'}%{end}", probably you intended:

<<EOS
{% if true %}
  Hello #{'World'}
{% end %}
EOS

or just double-quoted one of it.

I was listening a talk by Rasmus Lerdorf, the creator of PHP. He said that one of the most appealing feature he added to PHP was to include trivial templating out of the box.

But I don't know really how you guys feel about that, I am not a language designer. :)

I'm not a language designer too and probably this is Matz matter, but my personal feeling is that PHP's syntax is originally kind of template but Ruby is not. So templating is not Ruby language's responsibility and such work should be done by not syntax but a template engine as ERB has been kept as library.

And it's true, in Ruby or Python, it's harder to get started, you have to require an external library and you are overwhelmed by all these competing choices: Haml ? ERB (which variant?)

ERB is not "external library" and I recommend you to use standard library ERB if you want standard way for "trivial templating".
Disclaimer: I'm maintaining Haml and ERB.

You should write expected code in valid Ruby syntax. Guessing from "%{3.times do}Hello #{'World'}%{end}", probably you intended:

<<EOS
{% if true %}
  Hello #{'World'}
{% end %}
EOS

It was an idea that we could use a different notation: %{statement} instead of {% statement %} to be more similar to #{expression}

:)

[k0kubun (Takashi Kokubun)]

Ah okay, so expected feature is only "%{3.times do}Hello #{'World'}%{end}" and a multi-line complex usage would be:

<<EOS
%{if true}
  Hello #{'world'}
%{end}
EOS

k0kubun (Takashi Kokubun) wrote:

Ah okay, so expected feature is only "%{3.times do}Hello #{'World'}%{end}" and a multi-line complex usage would be:

<<EOS
%{if true}
  Hello #{'world'}
%{end}
EOS

Yes, that's it! :) I just wanted to improve string interpolation with %{statement} like syntax - and NOT adding a Template class or something like that (this was just a demo implementation in Ruby) :)

[shevegen (Robert A. Heiler)]

I was listening a talk by Rasmus Lerdorf, the creator of PHP.
He said that one of the most appealing feature he added to
PHP was to include trivial templating out of the box.

I do not think that templating was the killer feature of PHP.

I have not systematically analyzed the codebase of phpBB, mediawiki,
drupal or wordpress but I doubt that templating played a huge
role in any of these codebases.

in Ruby or Python, it's harder to get started, you have to
require an external library and you are overwhelmed by all
these competing choice

I somewhat agree with you here in the sense that inclusion into
stdlib is a good thing in general. That does not necessarily
mean that I agree with you specifically here in this example,
but in general, I concur there. I think that PHP has got only
one thing right and that was a focus on the www. I'd wish ruby
would have a similar focus on the www too. And I don't mean
external code bases such as rails; I really mean integral to
ruby itself.

But I don't know really how you guys feel about that, I am
not a language designer. :)

Ultimately you don't really need to care about everyone -
you only have to convince matz. :)

I mean, of course it would be better if you can convince others
in the ruby core team too, but ruby is matz's language and
he is the main designer.

I'm not a language designer too and probably this is Matz
matter, but my personal feeling is that PHP's syntax is
originally kind of template but Ruby is not.

I agree here with Takashi Kokubun; PHP had a different origin
and a different focus. Ruby has the better and more consistent
design everywhere in my experience.

On a general side note, no matter in this proposal but also not
with ERB or erubi or anything else - I always feel that all
those templating solutions are extremely ugly. I dislike all
of them. :)

That actually does not mean that I am against them per se, mind
you; I think it's perfectly fine that the FUNCTIONALITY is
available, be it in stdlib or in a gem - but from a beauty
aspect ... I really dislike all of them.

Yes, that's it! :) I just wanted to improve string interpolation
with %{statement} like syntax - and NOT adding a Template class
or something like that (this was just a demo implementation in Ruby) :)

I guess it is better because one '%' was eliminated in the process;
I still think that it is ugly, even after the elimination of '%'. :D

In the game wesnoth, they even used a XML variant as a programming
language, called WML; later they also added support for lua.

I have never before seen something as ugly as WML - you essentialy
would use conditional checks in something like [if condition] but
it looked like an abomination of a programming language embedded
into XML.

It is not related to your proposal at all of course but just for
comparison:

https://wiki.wesnoth.org/ReferenceWML

And more imporantly with examples:

https://wiki.wesnoth.org/SyntaxWML

XML-like tags to define variables.

[tag] key=value [/tag]

[parent_tag] key1=value1 [child_tag] key2=value2 [/child_tag][/parent_tag]

Anyway, ERB is already in stdlib so in theory it may be possible to
modify or target the behaviour there. I am sure ERB could allow for
multiple syntax behaviour too. You only have to convince matz in the
end. :)

https://ruby-doc.org/stdlib-2.4.1/libdoc/erb/rdoc/ERB.html

Yes templating is not a key factor for large modern PHP projects, but you need to go back in late 90s. That was the start of the web. A lot of people wanted to publish their things, so they wrote HTML pages. Often they knew nothing about programming, but it was so easy to enhance their website with PHP logic here and there. That's how PHP became largely popular and beat Perl for web use, because it was so simple to get started (yes the drawback was beginners doing a mess of spaghetti code). In Perl you had to use a CGI lib and to painfully concat html strings.

A lot of modern PHP frameworks built their own template engines, like this one called "Twig". Can you imagine that this thing is about 12 000 loc? I think it's overkill, it's like rebuilding a language on top of an other language. Why would you want to reach for a 12 000 loc library when plain old vanilla PHP already does that?

In its core, templating and text manipulation is just about 2 things: expressions and statements. Right now, Ruby only offers #{expression}, but it would be so cool to embed %{statement} too.

Yes I agree that templating may not be very beautiful visually, because you are mixing two languages like HTML and a backend language, but it's the most straightforward way of enriching documents with programming logic, without having to reinvent a new library (like WML) on top of your stack.

For the fun, I modified my original template code to make a Python-Django style template engine in Ruby: {% statement %} and {{ expression }} - this version is even faster and I got rid of the ugly "\0" string delimiter:

class Template

  attr_reader :input

  def initialize(input)
    @input = input
  end

  def output
    @input.dump.prepend('_=').gsub('{%', '";').gsub('%}', ';_<<"').gsub('{{', '#{').gsub('}}', '}')
  end

  def render(binding)
    eval(output, binding)
  end

end

[matz (Yukihiro Matsumoto)]

• Status changed from Feedback to Rejected

I think it should be done in the gem. Templating need not to be a part of the core.

Matz.