Combining Vim’s Syntax Definitions

If you fall on the vi side of the "Great Editor Debate," then hopefully you're using vim (http://www.vim.org). It's got all the features of vi and more. One powerful feature is called syntax highlighting; it helps immensely when programming because it provides additional visual clues when editing source code.

If you fall on the vi side of the “Great Editor Debate,” then hopefully you’re using vim (http://www.vim.org). It’s got all the features of vi and more. One powerful feature is called syntax highlighting; it helps immensely when programming because it provides additional visual clues when editing source code.

However, if you turn on syntax highlighting (by using syntax on in your local .vimrc file) and then attempt to view code that contains a mix of Scheme and Perl (as outlined in last month’s column), you’ll probably see something like what is shown in Figure One.

Bad Highlight
Figure One: Viewing a mix of Scheme and Perl in vim.

Because vim doesn’t know that everything within a pair of curly braces is Perl code, it highlights that section in red as if there were a syntax error. Vim can highlight either Scheme or Perl on an individual basis, but it can’t do both of them simultaneously. Not yet, anyway.

The Structure and Interpretation of Syntax Highlighting Files

Vim uses syntax highlighting files to control how the text from the file being edited should be displayed. The file name or suffix generally determines which highlighting file to use.

These syntax-highlighting files are usually composed of two major sections. In the first, syntax elements of your language are defined. This usually involves a lot of regular expressions.

In the second section, you take all the syntax elements that were defined and make them colorful by associating them with vim’s standard highlighting groups. Pretty much all but the most esoteric of vim’s syntax-highlighting files follow this structure.

Our example is no exception, but the nature of our hybrid language makes things interesting. Instead of defining syntax elements ourselves, the syntax elements of two existing syntax-highlighting files are going to be reused and combined. The new syntax will be called “scmpl,” and will be defined in a file called scmpl.vim (see Listing One, available online as http://www.linux-mag.com/downloads/2002-03/scmpl.vim).

Listing One: scmpl.vim

1 ” Scheme w/ Perl as a preprocessor
2 ” Language:Scheme+Perl
3 ” Maintainer:John Beppu, beppu@cpan.org
4 ” Last Change:2001 Dec 17
5 ” Location: http://www.linux-mag.com/downloads/2002-03/scmpl.vim
7 ” for portability
8 if version < 600
9 syntax clear
10 elseif exists(“b:current_syntax”)
11 finish
12 endif
14 ” load all of the scheme info
15 source $VIMRUNTIME/syntax/scheme.vim
16 unlet b:current_syntax
18 ” load all of the perl info into @Perl
19 syntax include @Perl $VIMRUNTIME/syntax/perl.vim
20 syntax region scmplPerl
21 \ start=/{/
22 \ end=/}/
23 \ contains=@Perl, scmplPerl
25 ” the script header
26 syntax match scmplSharpBang
27 \ “^#!/usr/bin/env.*gimp-request.*$”
29 ” link syntax elements to standard highlighting groups
30 if version >= 508 || !exists (“did_scmpl_syn_inits”)
31 if version < 508
32 let did_scmpl_syn_inits = 1
33 command -nargs=+ HiLink hi link <args>
34 else
35 command -nargs=+ HiLink hi def link <args>
36 endif
37 HiLink scmplSharpBang PreProc
38 endif
40 ” finish
41 let b:current_syntax = “scmpl”

Defining Syntax Elements

By convention, syntax highlighting files start with comments that provide information about the file. Lines 7 to 12 contain code that prevents this syntax-highlighting file from being loaded more than once. There are different ways of doing this in vim 5 versus vim 6, so an if-clause is necessary for portability. All syntax highlighting files should start like this.

On line 15, we move on to the actual meat of the syntax file. As mentioned before, we start with definitions of all the syntax elements of our language. We’ll first import the syntax definitions of Scheme by calling source syntax/scheme.vim.

The source command reads this file into our current context. This is like saying, “My syntax is a lot like Scheme’s.” However, it’s not exactly the same as Scheme so we have to unlet b:current_syntax on line 16. Later, we’ll assign b:current_ syntax with a more appropriate value, but right now, it will only get in the way of including Perl’s syntax highlighting file.

Unlike the way we brought in syntax/scheme.vim, the definitions found in syntax/perl.vim will be put into the variable @Perl instead of being executed in the current context (line 19). Then the areas of text where vim should expect Perl code to be found is defined by using the syntax region statement (lines 20-23).

Surprisingly, all the hard work is now done. Scheme will be highlighted properly at the top level, and code within pairs of curly braces will be treated as Perl. The only thing left to highlight is the first line of our scripts that start with:

#!/usr/bin/env gimp-request

This is taken care of with a simple syntax match statement on lines 26 and 27. Now that we’re done defining syntax elements, we can move on to adding colors.

Color Coordination

Power Tools (gvim-xpm)
Vim’s XPM Highlighting: Try opening an .xpm image using gvim (the GUI version of vim). This is syntax highlighting at its finest.

Instead of hard-coding colors, vim defines a set of standard highlighting groups. (From command mode in vim, type :help group-name to see a list of these highlighting groups.) In order to color a syntax element, it’s standard practice to associate it with a highlighting group. By not hard-coding the colors, vim users are given the freedom to define their own color schemes. It’s possible to associate syntax elements with hard-coded colors, but you had better have a good reason. (See the Vim’s XPM Highlighting sidebar for an example of a “good reason.”)

Anyway, back to the example. It defines only two syntax elements on its own and inherits a whole bunch from syntax/scheme.vim and syntax/perl.vim. Thankfully, we don’t have to concern ourselves with the inherited syntax elements, because they’ve already been colored. Out of the two syntax elements left, scmplPerl doesn’t need to be given color, because whatever is inside this region will be syntax-highlighted as Perl. (Combining two syntax files is a lot less work than you might think!) All that’s left is the syntax element scmplSharpBang, which could use some color; to do that, we have to define the HiLink command. Lines 30 to 36 do this in a portable way. (Just substitute every occurrence of “scmpl” for something else when writing your own syntax files.)

Figure Two: Viewing the same mix after combining syntax files.

It’s unfortunate to have to do all that work to define the HiLink command when we’re only going to use it once, but that’s life. If we were working on a more normal syntax-highlighting file, there would be a long list of HiLink commands around line 37. Anyway, everything that needs to be colored is now colored.

To finish this off, the syntax-highlighting file needs to state its name. On line 41, we say let b:current_ syntax = “scmpl”. All syntax-highlighting files end this way.

After installing scmpl.vim (see the Installation sidebar for instructions), you should be able to look at a file that mixes Scheme code and Perl code and see something like Figure Two .


The procedure for installing a syntax-highlighting file is not uniform across versions of vim. The steps below will show you how to install the files for either vim 5 or vim 6.

  • First, you have to find out what your $VIMRUNTIME is. Type :echo $VIMRUNTIME while in command mode to do this. This will be a directory like /usr/share/vim/vim60.

  • Next, copy your syntax file to the $VIMRUNTIME/syntax directory.

  • If you use vim 5, open $VIMRUNTIME/syntax/synload.vim and look for a long list of SynAu statements. Add a line that says:

SynAu scmpl

  • Next, we have to edit $VIMRUNTIME/filetype.vim. Find where Scheme is handled and replace that section with the following:

” Scheme
au BufNewFile,BufRead *.scm call SetFileTypeScheme(getline(1))

fun! SetFileTypeScheme(name)
if a:name =~ ‘gimp-request’
set ft=scmpl
set ft=scheme

  • If you use vim 6, you don’t have to mess with $VIMRUNTIME/syntax/synload.vim. All you have to do is change $VIMRUNTIME/filetype.vim so that the definition for Scheme looks like:

” Scheme
au BufNewFile,BufRead *.scm call SetFileTypeScheme(getline(1))

fun! SetFileTypeScheme(name)
if a:name =~ ‘gimp-request’
setf scmpl
setf scheme

The difference in vim 6 is that the SetFileTypeScheme function uses setf statements instead of set ft.

After doing all this, files ending with the .scm extension will be checked whether they are normal Scheme or Scheme that’s preprocessed with Perl and highlighted accordingly.

More Useful Than You Think

Mixing two or more languages together happens more often than you think. Anyone who has done Web development can attest to this. There are a million templating systems out there that mix HTML and some programming language into one file.

Whether you’re using Java, PHP, Python, Perl, ColdFusion, ASP, Ruby, or whatever, someone has come up with a way of embedding that language into HTML. Although our example uses Scheme instead of HTML as the base language, the concept of templating is the same.

If your templating system of choice is fairly popular, chances are that a syntax-highlighting file for it already exists. If that’s not the case, though, why not try making one? All the hard work has already been done — it’s just a matter of gluing syntax-highlighting files for existing languages together. If you do a good job, consider contributing it to the vim project by sending it to Bram Moolenaar (the author of vim) at Bram @moolenaar.net. Combining two syntax files is a lot less work than you think, and it’s a great way to contribute to Free Software.


Vim Online (Code and Tip Repository) http://vim.sourceforge.net

Vim’s Online Help (do this from command mode)

:help syntax
:help group-name
:help syn-include

John Beppu is addicted to Perl. He can be reached at beppu@cpan.org.

Comments are closed.