XML may be the best way to store your data, but how do you make it presentable? Enter XSL.
The story so far: you’ve got a mountain of data coded in XML, you’ve got schemas to make sure the data is valid, and you’ve written tools to manipulate the XML documents in a sensible way.
Now your boss has asked you to generate a report for the board meeting tomorrow. You know just what needs to be done: you have to convert your XML into something that can be displayed and printed–like HTML or a PDF file.
You could whip out your favorite programming language and write a tool to format the information, but data conversion tools are tedious to write. It would be nice if there was a better way. What you need is a tool that lets you describe how to transform your XML into “something else” and not have to write the code that does the transformation.
The Extensible Style Language (XSL) family of specifications can help save the day.
Making it Presentable
XSL makes it easy to change an XML document into a readable on-screen display or an attractive printed presentation (or another XML document based on a different schema).
Instead of writing your own tools, you can use XSL to define rules — change this element to that element, ignore this attribute, sort these sub-elements into this order, etc. — and use pre-built tools to interpret the rules and process the document.
XSL is composed of three complementary technologies:
- XSL Transformations (XSLT, http://www.w3.org/TR/xslt/) is a language for describing transformations. It allows you to describe how you want specific parts of your document transformed for presentation. XSLT transformations can produce XML, HTML, or any other type of output.
- XML Path Language (XPath, http://www.w3.org/TR/xpath/) is a language for addressing parts of an XML document. It allows you to select the sections of an XML document that you want transformed. In other words, if you picture an XML document as a hierarchy of elements, XPath lets you choose which element, set of elements, or levels of the hierarchy to transform.
- Extensible Style Language (XSL, http://www.w3.org/TR/xsl/), also known as XSL Formatting Objects (XSL-FO), is an XML-based language for specifying formatting semantics. In particular, XSL-FO allows you to specify the constraints for high-quality print formatting. XSLT is often used to produce a document composed of elements in the XSL-FO vocabulary.
Unfortunately, the acronym “XSL” is used to refer to the single Extensible Style Language specification as well as the collection of all three specifications. Even worse, the acronym “XSL” is also used to refer to various elements found in the XSLT language. In this article, we’ll always refer to XSL Transformations as XSLT.
Let’s take a closer look at XSLT and XPath in the context of two specific tasks: a transformation from XML to HTML and a simple XML-to-XML transformation. This article won’t cover transformation from XML to XSL-FO; however, once you understand XSLT, you can emit HTML, XML, or XSL-FO equally well. Likewise, it’s impossible to cover all of the features of XSLT and XPath in such a short space, but the topics covered here will get you started.
The XSL Processing Model
To transform an XML document to another form (e.g., XML, HTML, raw text, PDF, etc.) you need an XML document (the source), a set of transformation rules, and an application to interpret the rules, read and process the XML source, and emit a new representation of the data (the result).
In XSL, the set of transformation rules is collectively called an XSLT stylesheet. An individual rule is referred to as an XSLT template. The application that interprets the stylesheet and performs the actual transformation is called an XSLT processor.
You already have the source XML document, and there are several Open Source and commercial XSLT processors available: Saxon (http://saxon.sourceforge.net/), Xalan (http://xml.apache.org/), and xsltproc (http://www.xmlsoft.org/XSLT/), to name just two. Your work will focus on creating a set of templates to control the XSLT processor.
Every template has two parts: an XPath expression that selects elements in your source document, and a set of instructions to transform those elements to the target format. Valid XPath expressions are element names, such as “address“; paths to elements, such as “address/firstname” which specifies the element “firstname” contained in the element “address“; and wildcards such as “*“, which means “all elements.”
Some instructions in your stylesheet may be very simple: copy an element verbatim from the source to the result. Other template instructions might rename an element or condense a hierarchy of elements into a single element. Of course, you might want to ignore elements in the source XML. In that case, you can omit the template for that element or create a template without instructions (both can be done in XSLT, but which technique to apply depends on the transformation you want to achieve). Because templates can also emit whatever format you want (text, HTML, or XSL-FO elements) simply write the instructions to produce the target format.
How does the XSLT processor work? First it takes your source XML document and converts it into a tree composed of nodes. This is known as the source tree. (You can think of this tree as a Document Object Model (DOM) view of your source document; however, XSLT processors are not required to implement any specific object model. For more on converting an XML document into a tree, see “DOM-inating with XML” in the October 2001 issue, available online at http://www.linux-mag.com/2001-10/xmldom_01.html). The processor then uses your XSLT stylesheet, matching templates to nodes, performing a tree-to-tree transformation; the transformation yields what’s known as the result tree.
If you’re converting one XML document to another, the result tree may need no additional processing except to capture a text-based representation of the tree to a file. If you’re converting an XML document to HTML, the result tree (most likely a tree of HTML elements) might simply be sent to the browser. Or, when transforming an XML document into an XSL-FO “document,” the resulting XSL-FO tree would be passed directly to the next-stage XSL-FO processor to emit the tree as a PDF file.
|Illustration One: Sample address book document as a tree |
Converting XML to HTML
To start learning XSL we’ll create an XSL stylesheet to transform an address book stored as XML into HTML. (The full text of this stylesheet, and all the stylesheets discussed in this article, can be downloaded from http://www.linux-mag.com/downloads/2002-06) We’ll reuse the address book we created in the article “XML Schema Languages” in the February 2002 issue (http://www.linuxmagazine.com/downloads/xml_schema.tar.gz). The document will look like Figure One. Illustration One shows the same document as a tree.
Figure One: Sample address book document
<street>123 Any Street</street>
Most stylesheets begin with the <stylesheet> element (or its synonym, <transform>, used in the xsl namespace):
This stylesheet directive uses the xsl: prefix to identify elements in the XSL namespace, but as always with namespaces, the prefix is arbitrary. It also uses XSLT version 1.0, which is declared with the version attribute.
Next, we specify what the output should look like. The <xsl:output> element chooses what sort of serialization to use (XML, HTML, or text), the form of character encoding, whether elements should be indented, and a number of other parameters. To produce HTML output, we say:
<xsl:output method=”html” indent=”no”/>
The indent attribute tells the processor whether it should make the resulting XML “pretty” when it’s serialized. In general, it’s a bad idea to enable indenting if your output document isn’t pure data. For example, in the case of HTML, turning on indenting sometimes results in extra whitespace in <pre> elements, where white space could be significant.
As mentioned above, the heart of any stylesheet is its templates. To transform a document, the XSLT processor starts at the root of the source tree and performs an in-order travesal. For each node it encounters, the processor looks for a matching template in the stylesheet. If it finds a template, it follows the template’s instructions to construct a part of the result tree. Any node without a matching template is copied into the result tree as-is.
Figure Two shows a sample template. All templates starts with the <xsl:template> element. The match attribute specifies the type of node it should apply to. The “/” pattern is special: it matches the root of the document, which “occurs” before the first element. Thus, the “/” template is always processed first.
Figure Two: The root template
Normally, everything between the <xsl:template> and </xsl:template> elements is copied into the result tree. This includes elements which are not in the XSLT namespace, such as the HTML elements <html>, <head>, and so on. Any elements that are in the XSL namespace are usually XSLT template instructions; those are not copied, but are processed in the order in which they appear.
In Figure Two, <xsl:apply-templates> is an example of a processing instruction. <xsl:apply-templates> element tells the processor to interrupt its current traversal of the tree, and start a new “sub-traversal.” This sub-traversal begins at the current node and proceeds down through the sub-tree rooted at the current node in the same in-order fashion.
While the sub-traversal progresses, the processor continues searching for nodes to process and transform. Finding additional template matches along the way can result in more sub-traversals.
Once a sub-traversal finishes, the traversal it interrupted is restarted. However, when the processor resumes the interrupted walk, it will ignore all nodes traversed by the sub-traversal. In XSLT, every node in the source tree is examined only once.
An important concept to understand in XSLT is context. The “context node” is the node currently being processed. Many XSLT instructions change the context. More precisely, the processor always compares the context node to the list of available templates. The processing instruction <xsl: apply-templates/> tells the processor: “Select each child of the current context node in document order; for each one, make it the current context node; find the template that matches it; and process that template.”
The template in Figure Three matches <addressbook> elements in the source tree (in our sample document there is only one since <addressbook> is the root element of our simple schema). The most common type of match pattern is a simple element name like this one (“addressbook“). But match patterns can be more complicated. It’s possible to match elements only in a certain context or elements that satisfy certain criteria. We’ll see how to do that later on.
Figure Three: An <addressbook> template
<xsl:text>There are </xsl:text>
<xsl:text> addresses in the address book.
The template in Figure Three is only processed when the context node is <addressbook> (as defined by the <xsl: template> match pattern).
This template introduces a couple of new XSLT instructions. An <xsl:text> element simply wraps text. Using an explict wrapper avoids some whitespace-stripping issues. (For example, there’s a space after the word “are” that will not be stripped.)
The <xsl:value-of> element evaluates the expression given in its select attribute, converts the result to a string, and returns the string. count() is an built-in XSLT function. In this case, the expression “count(address)” counts how many children of the context node are <address> elements. So, this expression will count all the addresses in the address book. If there are four addresses in the address book, this template will insert the following text into the result tree.
<p>There are 4 addresses in the address
After processing the <xsl:text> and <xsl:value-of> instructions, we now process the <xsl:apply-templates> instruction. <xsl:apply-templates> suspends the current traversal and starts a sub-traversal over each of the children of the current context node. Here, the context node is <addressbook>. That node contains a number of <address> elements, so those are processed next. The template in Figure Four matches each <address> element.
Figure Four: An <address> template
This time, the <xsl:apply-templates> instruction has a select attribute. The expression given in select identifies the particular child nodes to process (recall the default selection is all the children of the context node).
The syntax of a select expression is governed by the XPath Recommendation. We’ll talk about XPath in just a little while. For now, we’ll use select expressions to simply list which elements we want to process. (Match patterns are also XPath expressions, but their syntax is limited to a subset of XPath.)
The first select expression, “name|pobox|street” chooses all of the <name>, <pobox>, and <street> elements, in the order that they occur in the document. The vertical bar in a select expression represents the union (“OR”) operator.
After all of the <name>, <pobox>, and <street> elements have been processed, the remaining <xsl:apply-templates> elements and select expressions select the <city>, <state>, and <zip> elements, respectively. (Technically, they select all of each element, but the schema dictates that there will only be one of each.)
The text between the state and the zip code,    , is two unbreakable spaces (the Unicode value for an unbreakable space is 160 decimal).
By placing our select expressions in a specific order, we force the matching elements into the corresponding order in the result tree. Even if the elements were “out of order” (assuming our schema allowed it), they would now be “in order” after processing.
Also notice that if we’d mistakenly written the first select as “name|street” (forgetting about pobox), then any <pobox> elements would be ignored. Once the sub-traversal is completed, the parent traversal will not traverse the elements which were ostensibly handled by the <xsl:apply-templates> instruction. A mistakenly skipped node is a bug in the stylesheet.
Still More Templates
There are two templates in Figure Five to match the address elements. The first matches <name>, <pobox>, and <street> elements. The body of this template processes the contents of each element by calling <xsl:apply-templates/>. This causes the text of each element to be copied directly into the result tree. Remember the text of each element is contained in a node in the source tree and the default is to copy any node for which there is no matching template.
Figure Five: Matching address elements
After that, an empty <br> element is placed in the result tree. Note that the XSL stylesheet is itself an XML document, so the proper XML “empty tag” syntax must be used (if the result tree is serialized as HTML, the proper “<br>” tag will be produced). Finally, the <city>, <state>, and <zip> elements are processed the same way as the previous three elements.
Figure Six shows the HTML result of processing the sample address book in Figure Five.
Figure Six: Address book formatted as HTML
<meta http-equiv=”Content-Type” content=”text/html; charset=utf-8″>
<title>Address Book</title></head><body><p>There are 4 addresses in the address book.</p>
<p>John Smith<br>123 Any Street<br>Anytown, MA 01004</p>
<p>Jane Smith<br>123 Any Street<br>Anytown, MA 01004</p>
<p>John Bigboote<br>314 Pi Ave<br>Grovers Mills, NJ 08648</p>
<p>Jane Doe<br>1 Missing St<br>Someplace, NY 10130</p>
After that quick walkthrough of XSLT, let’s take a closer look at some of the more technical details and examine how elements can be more precisely matched.
The general structure of an XSL stylesheet is a set of templates that describe how to turn “these things” in the source document into “those things” in the result document. To use XSLT, you must be familiar with both the source and the result: how to point to “these things” and how to describe “those things.” XPath is the language used to describe “these things.”
XPath was designed to be used in both attribute values and URIs, so it’s a string-based expression language. Most XPath expressions locate or identify elements in the source tree. Other, more general expressions are covered later in the section on predicates.
An XPath expression is known more formally as a location path, a set of one or more location steps separated by slashes. The slashes separate levels of hierarchy. Each step, in turn, is composed of three parts: an optional axis specifier, a node test, and an optional predicate. Although they are specified in that order, let’s take a look at the node test first, as the other two modify the meaning of the node test.
The node test chooses a specific kind of node based on its type (there are six node types: element nodes, attribute nodes, text nodes, comment nodes, processing instruction nodes, and namespace nodes). The simplest node test, and therefore the simplest location step and location path, is just the name of an element (e.g., address): it selects element nodes with only that name (e.g., <address>). Some other examples are:
- * selects element nodes.
- prefix:* selects element nodes that use the namespace prefix prefix. More accurately it selects element nodes that are using the same URI associated with the prefix prefix, even if some of the nodes are using a different prefix. For example, if prefixes a and b both referred to the same namespace URI, then a:* would also match all element nodes in the b namespace.
- text() selects text nodes.
- comment() selects comment nodes.
- processing-instruction() selects processing instruction nodes.
- processing-instruction(‘xxx‘) selects processing instruction nodes that have the target “xxx“.
- node() selects nodes of any type.
Once you’ve specified a node test, the next thing to qualify is the location, in relation to the context node, where the node test should apply. That’s the role of the axis specifier. An axis specifier acts as a filter by identifying a segment of the document tree and restricting the node test to match nodes only in that segment. The axis specifier precedes the node test and is separated from it with a double colon. There are seven axes in XPath that completely subdivide the tree:
- self selects the current context node. The context node is always “self::node()“, which can be abbreviated “.“.
- ancestor selects ancestors: the parent node, and all of its partent’s, recursively.
- descendant selects descendants: all of the children of the context node and all of their children, recursively.
- preceding selects preceding elements. If you think of a tree of element nodes, this axis selects all the elements whose end tag occurs before the start tag of the context node. More generally, it’s all nodes that started and ended before the context node started.
- following selects following elements: all nodes that start after the context node ends.
- attribute selects attributes. (If the context node is not an element node, this axis selects nothing.) The attribute axis can be abbreviated “@“. For example, “attribute::class“, which selects the class attribute of the context node, can be abbreviated “@class“.
- namespace selects namespace nodes. Namespace nodes associate namespace prefixes with namespace names.
There are six additional axes that will select some other useful subsets of the tree.
- child selects the children of the context node. This is the default axis. This is not recursive (in the way that descendant is).
- parent selects the parent. The parent node, “parent:: node()” can be abbreviated “..“. This is not recursive (in the way that ancestor is).
- preceding-sibling selects the preceding nodes that are siblings of (have the same parent as) the context node.
- following-sibling selects the following nodes that are siblings.
- ancestor-or-self selects ancestors and the context node.
- descendant-or-self selects descendants and the context node.
One important aspect of an axis specifier is the order in which it returns nodes. In general, nodes are returned in the order they appear in the document. The exceptions are preceding, preceding-sibling, ancestor, and ancestor-or-self. These return nodes in reverse document order.
Consider our earlier example of John Smith’s address. In the template that matches <city>, following-sibling::* would return <state> and <zip>, in that order. But if you asked for preceding-sibling::*, you’d get <street> and <name>, in that order.
The combination of an axis specifier and a node test selects all the nodes of a specific type from a specific segment of the source tree. But sometimes you need even finer control over the selection of nodes. This is the purpose of the predicate.
A predicate is an XPath expression in square brackets following a node test. The simplest predicate is a number: it selects the nth node. For example, the following expression selects the second <para> child of the context node:
When using predicates with axis specifiers that return nodes in reverse document order, the lower the number, the closer the node is to the context node. For instance, preceding-sibling is the closest immediate preceding sibling, preceding-sibling is the next one further away, and so on.
In general, the XPath expressions that occur in the predicate can be arbitrarily complex. Take a look at this expression:
or @mark != "bullet"]
and position() > 4
and size() < 20)]
This selects all of the listitem elements that are among the descendants of the context node that also satisfy the following additional constraints:
- They have a list element parent that either has no mark attribute or has a mark attribute that is not equal to “bullet“.
- Their position in the list is greater than 4. The position() function returns the position of a node within its context: for elements, this is its child number in document order.
- The total number of elements in the list is less than 20. The size() function returns the number of nodes in the context.
position(), size(), and not() are other XSLT functions.
When a predicate is being evaluated, the context node is temporarily set to the node that matches the node-test with which the predicate is associated.
As specified earlier, location steps (an axis specifier, node test, and predicate) can be strung together to form location paths. Location paths can be used to choose nodes that satisfy all of the conditions of each step. For example, the following expression selects all of the <title>s in <figure>s that occur in<chapter>s below the context node.
The single slash selects one level of hierarchy. On the other hand, a double slash (“//“) indicates arbitrary levels of hierarchy. The following expression selects all the titles of figures that occur at any level of chapters below the context node.
For example, in Figure Seven , if the context node is the <book> element, the first expression above would select only the “F1″ title (“E1″ is not a figure title and “F2″ and “F3″ are not the titles of figures that are the direct children of <chapter>). The second expression would select both “F1″ and “F2″ (but not “E1″ or “F3″).
Figure Seven: A very short book
If a slash occurs at the beginning of an expression, it selects the root of the tree. Referring to Figure Seven, the expression “/book” would select the <book> element regardless of the context node.
If a double slash occurs at the beginning of an expression, it potentially selects every node in the tree. Use this expression carefully, as it can have a significant performance impact on the transformation process.
Transforming XML to XML
Another common task for XSLT is to transform one XML document into another. For example, suppose that you wanted to provide explicit markup for first and last names in the address book. This will make it easy to sort the address book entries. Figure Eight shows the desired result of the transformation. The relevant XSLT constructs are shown in Figure Nine. This is an XML to XML transformation, so the output method is xml.
Figure Eight: Desired XML output
<street>123 Any Street</street>
Figure Nine: XML in, XML out
<xsl:output method=”xml” indent=”no”/>
“substring-before(., ‘ ‘)”/></firstname>
“substring-after(., ‘ ‘)”/></surname>
As is often the case in XML to XML transformations, most of the XML elements will be copied through without change. This is in contrast to our previous XML to HTML transformation, where we didn’t want any XML elements to pass through to the final HTML document (since they’d likely be illegal tags in HTML).
The first template performs the task of copying XML elements. The match pattern “*” matches any element. It’s then copied via the <xsl:copy> instruction. The <xsl:copy> instruction can copy any type of node. It automatically copies namespace bindings, but does not copy attributes or children of elements.
The <xsl:copy-of> instruction is similar to <xsl:copy>, but it allows you to select specific nodes. In this case we have a select pattern that copies attributes. Just as “*” matches all elements, “@*” matches all attributes. This allows us to copy all the attributes.
Finally, the <xsl:apply-templates> instruction makes sure the content of the element is copied and copies all its children.
Now, if this were the only template in the stylesheet, the result of the transformation would be a complete logical copy of the source document. However, there are some things, such as whitespace in attribute values and entity references, that cannot be preserved with XSLT. You cannot write an XSLT transformation that will always produce a byte-for-byte identical result document.
Our goal here, though, is to make changes to the structure of names, so next is a template for <name> elements. A template with an explicit name always has a higher priority than a template that matches “*“, so this template will be used for names.
This template uses <xsl:copy> and <xsl:copy-of> to make a copy of the <name> element and its attributes. But instead of applying templates to create the internal structure, <firstname> and <surname> elements are created.
The substring-before() and substring-after() functions, as their names suggest, return the text of a string before and after a separator character. So, given the name “John Smith” as the context node (the dot, “.“), “substring-before(.,’ ‘)” will return the string before the first space, “John“. The last name is similarly extracted with the substring-after() function. The result of this stylesheet will be a copy of the original address book, with updated markup for names.
Of course, if the address book contains people’s middle names, simply chopping the name in two at the first space isn’t going to give very good results. You’ll have to do some cleanup by hand. It’s difficult to automatically add more markup to a document, but it’s easy for the processor to ignore markup. As a general rule, always create data with as much markup as you think you’ll ever need, and then add a bit more. Data that isn’t marked up isn’t accessible.
The ability to transform XML into almost any form makes XML a much better way to store data. With the capabilities of XPath and XSLT, any data stored as XML can be presented in almost any form desired. For instance, a single Web page in XML could be presented to the client in whatever form of markup it’s capable of handling. That means the same Web page needn’t be placed in separate files (an HTML version, a WML version, etc.).
To learn more about XSL (especially XSL-FO) read the W3C Recommendations or any of the numerous books available which cover everything from from the basics (such as named templates, numbering, and conditional processing) to considerably more advanced topics (such as extension functions and elements). However, with what we’ve covered here, you can begin writing simple transformations on your own. And in no time an all, you’ll be able to get your boss that presentation he needs for the board meeting.
Norman Walsh is a staff engineer at Sun Microsystems. He participates in a number of standards efforts worldwide, including the OASIS DocBook Technical Committee, which he chairs. He can be reached at firstname.lastname@example.org.