The Template Toolkit, Part One

In some of my past columns, I've mentioned that my template system of choice is the aptly named Template Toolkit, a marvelous work by Andy Wardley. Although I've demonstrated how I've used the Template Toolkit (TT), I haven't really talked enough about what makes it so wonderfully useful. So, this month, let's take a more in-depth look at the wonders of TT.

In some of my past columns, I’ve mentioned that my template system of choice is the aptly named Template Toolkit, a marvelous work by Andy Wardley. Although I’ve demonstrated how I’ve used the Template Toolkit (TT), I haven’t really talked enough about what makes it so wonderfully useful. So, this month, let’s take a more in-depth look at the wonders of TT.

Why Template Toolkit?

At a minimum, a templating system provides a way of replacing placeholders with values. The simplest templating system is a Perl “one-liner” such as this:

my %v = (first => ‘Randal’, last => ‘Schwartz’);
my $text = ‘My name is <first> <last>.’;
## here it comes:
$text =~ s/<(\w+)>/$v{$1}/g;

Here, each word in angle brackets is replaced with a value found in the %v hash. Simple enough, and some might say deceptively simple. Because it’s this easy to code a trivial templating system, many people have started here and grown their own templating systems independently. But the tricky parts are indeed tricky. Eventually, templating systems grow to include features that make them approach full-blown languages.

But here’s where I think Andy did the right thing: TT’s control structures are handled by a mini-language. This mini-language carefully hides the differences between data structure access, method calls, and function calls, by sweeping them all under the same dot notation (just like Perl version 6). Thus, the TT mini-language is much more accessible to web designers who don’t care to learn the full-blown intricacies of Perl element access and method calls.

Additionally, the TT mini-language has just enough features to handle templating, but not quite enough to do serious heavy lifting. This helps me to do the right thing in the right places, because I sense a steadily increasing difficulty when I’m using TT code where full Perl code is really needed. For example, in a web application, the TT code is used for the View code of the Model-View-Controller triad, whereas Perl works better for the Model and Controller parts.

Another TT advantage: it was driven early in its development by two key projects: the (sadly now-defunct) etoys.com site and slashcode.com, the code behind Slashdot and hundreds of other web-based communities. Key developers from both projects provided valuable feedback to Andy about real world issues and concerns, guiding Andy in further design and features.

The TT community is also quite active, with the mailing list getting one or two dozen emails a day. Novice questions are welcome, particularly because such questions occasionally point out shortcomings in the language or documentation that we “experts” overlook automatically.

The TT mini-language is compiled into Perl code, which is then compiled and loaded into memory for execution. The Perl source code can be cached to disk, while the compiled subroutines can be cached in memory. Additional heavy-lifting code can be written directly in Perl, and loaded as modules with nice TT interfaces. For prototyping, you can also embed Perl directly in your templates, if you prefer. This all works together to ensure fairly speedy execution, even for traffic-intensive web sites.

Almost every part of TT is configurable, sometimes excessively so. If you want a slightly different grammar for your TT mini-language, you can plug that in. If you want to load your templates from a database instead of a file, you can have that too. If you want your scalars to know how to rot13 themselves, you can get that added. Perhaps this is the source of the “Toolkit” part of the name: what you’re really getting for your application is a templating system built to your specifications from the large class of templating systems that TT supports.

The TT Language

The TT language consists of directives and variables. Directives provide the control instructions, like IF and WHILE. Variables map directly to Perl scalar, array, hash, and object variables. The TT code can be structured by having templates include other templates in various ways, similar to subroutines in a traditional language.

Let’s look at a sample template:

Dear [% name %],

It has come to our attention that your account is in arrears to the sum of
[% debt %]. Please settle your account before [% deadline %].

This template contains three directives that interpolate the TT variables as indicated. (We’ll see later how the variables get their values.) The construct [% variable %] or [% GET variable %] (The GET is optional and is frequently omitted) interpolates the value of variable. Meanwhile…

[% SET variable = some + calculation %]

… sets a value into a variable. (The SET is also optional.)

If the [% and %] tags conflict with desired data, they can be changed to nearly arbitrary start and end sequences, such as HTML comment markers. Whitespace between the tags is almost entirely ignored. Comments start with # and extend to the end of the line.

By default, newlines are kept, so…

Hello [% a = 3 %]
World [% a %]

… results in Hello \nWorld 3\n. You can absorb the whitespace by placing - next to either %, as in…

Hello [% a = 3 -%]
World [% a -%]

… which results in Hello World 3. A configuration mode called post chomp treats all directives as if they have this trailing minus, which I find very useful and consistent.

Many control directives like FOREACH, WHILE, and IF nest as a block, terminating at a corresponding END. The output of a block directive may be captured into a variable:

[% result = FOREACH user = userlist %]
one user is [% user.name %]
[% END %]

Here, the output from the FOREACH loop ends up in result for later processing.

Directives can be semicolon-separated within a tag pair, which saves an adjacent end-tag/start-tag combination:

[% result = FOREACH user = userlist %]
one user is [% user.name; END %]

Some directives can trail other directives, similar to Perl:

[% b = b * a FOREACH a = [1, 2, 3] %]

Expressions are fairly Perl-like, supporting literal strings with ‘single quotes’ and “double quotes with $variable interpolation”. Like Perl 6, the concatenation operator is _ (“underscore”), not the dot.

The INSERT directive brings in another chunk of text, often from another file:

INSERT myfile

For INSERT, the contents of myfile are not processed for directives. The file is found along the INCLUDE_PATH, selected by a configuration parameter.

The INCLUDE directive is like INSERT as it brings in other data…

INCLUDE myfile

… but the contents of myfile are further examined for TT language constructs, thus making INCLUDE more like a subroutine call. Additionally, the argument might also be a named block in the same file, providing for local, common code for re-use:

[% BLOCK myfile %]
… some text here, with [% first %] [% last %] names.
[% END %]

Currently defined variables are visible to the included block or file, but variables set in the included section are not visible back to the rest of the file. For convenience, extra local variables can be defined during the invocation…

[% INCLUDE myfile this = "that" %]

… which provides a nice way to pass parameters down to the sub-template.

A PROCESS directive acts like INCLUDE, but the local variables remain in effect, thus sharing the same namespace as the invoker. Typically, these directives are used for speed (localization costs time) or for common variable initialization.

The WRAPPER directive…

[% WRAPPER foo %]
…some stuff…
[% END %]

… acts as if you’d said:

[% INCLUDE foo content = "some stuff" %]

This is great for writing text that wants to wrap some other text with enclosing materials:

[% INCLUDE comment_label WRAPPER my_button color='blue' %]

Here, the contents of include file comment_label can be enclosed in text provided by my_button, possibly modified by the current value of color.

The output of a block can be captured:

[% disclaimer = BLOCK %]
Portions of tonight’s show not affecting the outcome were edited.
[% IF sorry %]
We’re sorry.
[% END %]
[% END %]

This is a good way to define a large text string for boilerplate to use later.

The TT language provides a Perl-like IF structure:

[% IF age < 10 %]
Hello [% name %], does your mother know you’re using her AOL account?
[% ELSIF age < 18 %]
Sorry, you’re not old enough to enter (and too dumb to lie about your age)!
[% ELSE %]
Welcome [% name %].
[% END %]

And not to be outpaced by Perl 6, TT also provides a SWITCH structure:

[% SWITCH myvar %]
[% CASE value1 %]
that value
[% CASE [value2 value3] %]
either of those
[% CASE myhash.keys %]
any of those keys
[% CASE %]
[% END %]

The FOREACH loop acts like the Perl equivalent:

[% FOREACH thing = [foo 'Bar' "$foo Baz" ] %]
* [% thing %]
[% END %]

When iterating over a hash, omitting the iteration variable causes the keys to be assigned directly as variables:

[% userlist = [ { id => 'merlyn', name => 'Randal' }
{ id => 'fred', name => 'Fred Flintstone'} ]

[% FOREACH userlist %]
[% id %] is [% name %]
[% END %]

This loop acts like…

[% FOREACH u = userlist %]
[% u.id %] is [% u.name %]
[% END %]

… but with less typing. Nested loops are also supported, as are Perl-like NEXT and LAST operations.

Unlike Perl’s equivalent, TT’s FOREACH directive understands where it is with regard to the loop, and provides meta-information via the loop variable. For example, loop.size gives the total iterations, and loop.last is true if this is the last item. Using these controls, we can make the loops act nicely.

For example, this template…

[% FOREACH i = [ 'foo', 'bar', 'baz' ] %]
[% IF loop.first %]<ul>[% END %]
<li>[% loop.count %] of [% loop.size %]: [% i %]
[% IF loop.last %]</ul> [% END %]
[% END %]

… generates:

<li>1 of 3: foo
<li>2 of 3: bar
<li>3 of 3: baz

Without the loop controls, we’d typically move that start and end tag outside the loop, but then we’d get the tags even if the list was empty. Here, we’ll get start and end tags only if we’ve entered the loop at least once. Very cool.

The WHILE directive provides the expected “loop as long as an expression is true.” NEXT and LAST work as they do in Perl:

[% WHILE total < 100 %]
[% total %] [% total += 1 %]
[% END %]

But beware: to prevent runaway programs, all loops are limited to 1000 iterations (an arbitrary value selected by Andy).

Well, I’ve run out of room already, and I still have a bit more to say. Next time, I’ll finish up the descriptions of the directives, talk about exception handling, data structures, and using TT from Perl. I’ll also cover configuration directives, the command-line tools that come with TT’s distribution, and using TT with mod_perl. Until next time, enjoy!

Randal Schwartz is the chief Perl guru at Stonehenge Consulting. He can be reached at merlyn@stonehenge.com.

Comments are closed.