Go Directly to Jail

Available on all Linux and Unix systems, chroot jails can secure untrusted applications and make trusted ones almost impenetrable. Here's how to build them.

“Security” is one of those “buzzwords du jour,” and there seems to be as many approaches to security as there are opinions on Microsoft. However, unlike other hot topics (or “CEO hot buttons”) that come and go, effort spent on security almost always pays off. Moreover, having a multitude of security techniques is a very good thing. There are umpteen ways to hack a system, and the savvy system administrator maintains a substantial and varied arsenal of countermeasures. Firewalls, honey pots, intrusion detection, and SSH are just a few tricks of the Linux security trade.

Application jails, also known as “change root jails” or “chroot jails,” are another effective countermeasure. Supported by all Linux and Unix systems, application jails put up a nearly impenetrable barrier between the “jailed” software and the rest of the system. And because a jail is enforced by the operating system and not by an application, it can provide an enormous level of safety. A chroot jail “incarcerates” untrusted applications, and acts like a guard, almost literally, for applications that already have substantial security measures built-in.

This month, let’s learn about jails. Let’s throw an application into “solitary,” and make it a model citizen.

Uprooting Root

The chroot() system call has been with UNIX since at least Version 7 (released in 1979). As its name implies, chroot() changes the root directory of the calling process.

What does that mean? Think of chroot() as a kind of reality distorter. Once a running process executes chroot(“/home /jail”), /home/jail becomes “/,” and for all intents and purposes, every file and directory outside of /home/jail (including the true root directory and true /home directory) no longer exist.

In effect, chroot() provides a UNIX-inside-of-UNIX environment — a kind of “jail” where a process can be restricted to an arbitrary portion of the filesystem. Jails provide “security by default” for untrusted software: even if the software proves to be insecure in ways not anticipated, the jail (which is enforced by the operating system, not the program) dramatically limits the damage that can be done. Indeed, a jail should provide only a bare minimum of facilities, thereby limiting the potential for damage even further.

Let’s start by building a practice jail. While our practice jail won’t be very useful per se, building it will demonstrate the techniques needed to jail more complex applications (which we’ll tackle momentarily).

All examples in this article are based on Red Hat Linux 6.2. Some specifics — especially shared library names — are likely to be different on other releases of Red Hat and other versions of Linux and Unix.

Your First Jail Sentence

All process jails require “infrastructure” — an assortment of support files such as devices, shared libraries, system configuration files, utilities, and other programs — to make the jail self-sufficient. A jail’s specific infrastructure depends largely on the application being jailed.

For example, if you’re building a jail for a daemon, the jail won’t need basic Linux commands like ls and rm. However, if you’re building a jail to provide secure, remote access to a machine to an end-user, or are building a jail for a set of complex shell scripts, you’ll probably populate that environment with a more comprehensive set of utilities.

Let’s get started. Assuming you have a /home directory, create the directory /home/jail, and execute the following commands to build the framing for the jail:

# cd /home/jail
# mkdir bin lib dev tmp
# chmod a=rwx tmp
# cp /bin/bash /bin/ls bin

Since /home/jail will eventually be the root directory, we’ve recreated some of the common subdirectories you find in root: bin, lib, dev, and tmp. And like the real /tmp, we’ve made our jail’s tmp accessible to every user and process. We also copied bash and ls to the jail’s bin directory. However, those two commands aren’t usable from within the jail — yet.

Remember that most Linux commands depend on one or more shared libraries to run, and that’s the case with bash and ls. Those commands won’t work unless we copy the correct shared libraries into the jail, too. Which shared libraries do bash and ls depend on? Use the ldd command to find out.

# pwd
# ldd bin/*
libtermcap.so.2 => /lib/libtermcap.so.2 (0×40016000)
libc.so.6 => /lib/libc.so.6 (0x4001b000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×40000000)
libc.so.6 => /lib/libc.so.6 (0×40016000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×40000000)

As it turns out, bash and ls depend on three unique libraries: libc.so.6, ld-linux.so.2, and libtermcap.so.2. Let’s copy those libraries into the jail:

# cp /lib/libtermcap.so.2 /lib/libc.so.6 /lib/ld-linux.so.2 lib/

(Note that while ldd reports the full pathname of the required shared libraries, each hierarchy need not be replicated exactly in the jail. Putting all shared libraries in the jail’s lib directory is usually sufficient.)

In addition to its own set of shared libraries, the jail also needs a few devices (see the sidebar “Replicating Devices” for more information on device creation). The commands in Figure One create the null and zero devices in the jail.

Figure One: Populating the jail with two basic devices

# ls -l /dev/null /dev/zero
crw-rw-rw- 1 root root 1, 3 May 5 1998 /dev/null
crw-rw-rw- 1 root root 1, 5 May 5 1998 /dev/zero
# pwd
# mknod dev/null c 1 3
# mknod dev/zero c 1 5
# chmod a=rw dev/null dev/zero

Our jail is minimal, but it’s ready to go. Use the chroot command to enter the jail. chroot takes two arguments: a directory and a command. chroot runs the command with the root directory set to the specified path. See Figure Two.

Figure Two: Use chroot to enter the jail

# chroot /home/jail /bin/bash
# pwd
# ls -l
total 16
drwxr-xr-x 2 0 0 4096 Oct 29 16:56 bin
drwxr-xr-x 2 0 0 4096 Oct 29 20:01 dev
drwxr-xr-x 2 0 0 4096 Oct 29 16:56 lib
drwxrwxrwx 2 0 0 4096 Oct 29 22:41 tmp
# pwd

chroot /home/jail /bin/bash launches a new process with /home/jail as the new effective root. The command, /bin/bash, is subsequently launched relative to the new root.

Replicating Devices

Replicating device files in the jail requires a bit more than just a cp command. We’ll touch on device creation here. As a matter of habit, we create dev/null and dev/zero in all jails, as these are safe and used often enough that we don’t even research whether they are needed or not.

In the UNIX and Linux kernels, devices are known not by name, but by type (character or block) and by major and minor device number. These numbers are indexes into kernel tables that select a device’s handler functions. The name as appears in the filesystem serves as the entry point for user programs.

When creating a required device in the jail, always examine it in the full system first. For instance, if /dev/random were required, we’d run the ls command with a full listing to show the type and numbers:

# ls -l /dev/random (looking in the full system)

crw-r–r– 1 root root 1, 8 May 5 1998 /dev/random

Here, “c” on the left represents a “character” device (sometimes known as a “raw” device), and “1, 8″ are the major and minor device numbers, respectively.

To replicate this in the jail, we use the mknod command, which is standard in all UNIX/Linux systems, and takes the device name, type, and the two numbers as parameters:

# cd /home/jail

# mknod dev/random c 1 8

Block devices have type “b”, and they are much less commonly found in a jail. See the mknod manual page for full info on this command, including other more exotic device types.

As you can see, once inside in the jail, the rest of the filesystem is “invisible.” You can cd .. all day and never get outside the jail, and ls and bash are the only Linux commands available.

But notice that ls shows zeros for user name and group. Why? There are no users defined inside the jail, so the user ID to user name mapping fails, and the numeric user and group IDs are reported instead. You might guess that /etc/passwd is needed, but that’s only the start. The mapping from a user ID to a user name requires a substantial number of libraries — more infrastructure that must be added to the jail. Let’s track down those details.

Characterizing a Jailed Process

As mentioned above, most Linux programs required shared libraries to run. ldd can examine an application and discern which shared libraries are required for loading and launching the program. However, some applications load libraries dynamically. In some cases it’s impossible to identify all of the libraries that are required without actually running the program. That’s where strace comes in.

According to the strace man page, strace “… intercepts and records the system calls which are called by a process. The name of each system call, its arguments, and its return value are printed on standard error or to the file specified with the -o option.”

By tracing an application as it runs, we can discover all kinds of subtle and “dynamic” dependencies. Indeed, we must be able to generate a complete list of dependencies to successfully run an application in the jail. And before we can trace an application in the jail, strace itself must be moved into jail. Using the same technique applied to bash and ls, we copy strace into jail.

# cd /home/jail
# type strace
strace is /usr/bin/strace
# cp /usr/bin/strace bin
# ldd bin/strace
libnsl.so.1 => /lib/libnsl.so.1 (0×40016000)
libc.so.6 => /lib/libc.so.6 (0x4002d000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×40000000)
# cp /lib/libnsl.so.1 lib/

strace requires two of the shared libraries already in the jail, but also needs libnsl. The cp command copies libnsl into the jail. Now strace is ready to go.

Again, use the chroot command to enter the jail, adding the -o switch. -o is needed because the output of strace is so large that we need to capture it in a temporary file. See Figure Three.

Figure Three: Capturing the output of strace

# pwd
# chroot /home/jail strace -o /tmp/trace.out ls -l
execve(“/bin/ls”, ["ls", "-l"], [/* 23 vars */]) = 0
brk(0) = 0x80549b0
open(“/etc/ld.so.preload”, O_RDONLY) = -1 ENOENT (No such file or directory)
open(“/etc/ld.so.cache”, O_RDONLY) = -1 ENOENT (No such file or directory)
open(“/lib/i686/mmx/libc.so.6″, O_RDONLY) = -1 ENOENT (No such file or directory)
stat(“/lib/i686/mmx”, 0xbffff6a0) = -1 ENOENT (No such file or directory)

# pwd
# vi tmp/trace.out

Note that the output file is created relative to the root of the jail. After chroot finishes, we consult /home/jail/tmp/trace.out to see what happened.

If you open trace.out, you should see every system call made by ls. (Users of UNIX System V or Solaris may recognize this as the same function provided by the truss program.) Scan the file and look for file operations — especially open() system calls — that have failed. Failed operations generally have a “-1″ return code with ENOENT (no such file or directory) or EPERM (permission denied) error codes. Whenever we find an “important” file missing, we have to copy that file into the jail. Deciding what constitutes “important” requires a little bit of experience and some good detective work.

By the way, it’s common to see the same file requested from multiple directories (a kind of internal search path), and for the time being you can ignore these files: ld.so.preload, ld.so.cache, locale.alias, and LC_MESSAGES.

However, the following failed operation is curious and worthy of a special note:

open(“/dev/null”, O_RDONLY|O_NONBLOCK| O_DIRECTORY) = -1 ENOTDIR (Not a directory)

We added dev/null to the jail, but this request oddly requires that it be a directory. In this case, the failure is expected, so we have nothing to change.

A bit later in the trace listing we get to the first “interesting” part:

open(“/etc/nsswitch.conf”, O_RDONLY) = -1 ENOENT (No such file or directory)

The “name service switch” configuration file is at the center of the system databases, and entries in this file determine whether (say) user information comes from the traditional /etc/passwd file or from some other facility such as NIS. The default nsswitch.conf file is suitable, but we’ll create a smaller version tailored just for the jail.

# pwd


# mkdir etc

# cp /etc/nsswitch.conf etc

# vi etc/nsswitch.conf

trim down the file…

# cat etc/nsswitch.conf

passwd: files look in /etc/passwd

shadow: files look in /etc/shadow

group: files look in /etc/group

hosts: files dns look in /etc/hosts then DNS

networks: files look in /etc/networks

protocols: files look in /etc/protocols

services: files look in /etc/services

Once the file is in place, we rerun the trace. Now, you should see that /etc/nsswitch.conf opens correctly, but libnss_files.so.2 cannot be found. Unlike the previous shared libraries that were bound to the program at program-start time (and reported by ldd), libnss_files.so.2 is dynamically loaded only when it’s actually used. (This particular library processes the “files” parameters from nsswitch.conf). After copying libnss_files.so.2 into the jail, we run another trace. See Figure Four.

Figure Four: Tracking missing files

# cp /lib/libnss_files.so.2 lib
# chroot /home/jail strace -o tmp/trace.out ls -l
# vi tmp/trace.out
open(“/lib/libnss_files.so.2″, O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=247348, …}) = 0
read(4, “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\33″…, 4096) = 4096

close(4) = 0
open(“/etc/passwd”, O_RDONLY) = -1 ENOENT (No such file or directory)

open(“/etc/group”, O_RDONLY) = -1 ENOENT (No such file or directory)
open(“/etc/localtime”, O_RDONLY) = -1 ENOENT (No such file or directory)

Now that libnss_files.so.2 is present, attempts to read from other system files (including passwd, group, and localtime) fail. As before, those system files must be copied into the jail. However, rather that copy the real passwd and group files into the jail, we’ll create minimal ones. We don’t need the full complement of users or groups, and in no situation should you ever put an encrypted password into the jail.

# pwd
# vi etc/passwd
# vi etc/group
# cp /etc/localtime etc

Now, run the jailed ls again.

Success! The user and group names show up, and the last-modified time of each directory reflects the local time zone instead of UTC (courtesy of the jailed etc/localtime file).

You might think that this is a lot of work just to get ls working, but that mischaracterizes what we’ve accomplished. Rather than just support a single command, we’ve created an infrastructure that supports many other programs as well. Nearly every Linux jail will require at least this same setup, and adding the next command may very well require no additional effort.

Having said that, some additional effort will be required when networking support is added, and we’ll leave that research as an exercise for the curious reader. As a starting point, add the telnet command. (And here’s a hint: the dependencies are related to domain name service support.)

To Jail or Not to Jail

The simple jail created above is useful for introducing the core chroot concepts, but it’s not terribly useful in practice. In particular, it runs as root — that’s extremely dangerous because root can easily escape from jail (see “Busting Out of Jail” for more information). More broadly, however, it’s “just a shell,” and not a real application.

Busting Out of Jail

Though chroot provides enormous isolation, it’s not completely impenetrable. If the program in the jailed area has or can get root, it may be possible to break out of the jail and access the larger filesystem.

One of the techniques subverts chroot outright, while the others are indirect.

The traditional way to break out of jail is with a bit of C or Perl code (or perhaps even with a buffer overflow via an inbound network connection), and it involves changing root out from under the jailed process. Briefly described:

  • open(“.”) the current directory, retaining a file descriptor to it.

  • Use mkdir() to create a temporary subdirectory

  • Use chroot() to change to the new subdirectory, which “lowers” the root

  • Use fchdir() to change to the “saved” directory via the file descriptor. This puts the current directory of the process above the root.

  • Use chdir(“..”) repeatedly until the current directory is at the top of the filesystem

  • Use chroot(“.”) to raise the root to the very top

This requires root permissions.

A few other ideas for breaking out of jail, most of which require root permissions, are presented here. This is by no means an exhaustive list of approaches:

  • Use mknod to create a device file inside the jail that references the live filesystem (say, /dev/hda1, which maps to the root filesystem on my Linux box). By writing to the filesystem directly, it may be possible to add programs to the jailed area (say, a setuid root binary). This requires root privileges to run.

  • Use mknod to create a local /dev/mem device file and use it to modify kernel memory. This requires root privileges to run.

  • Find an important file that is shared between the full system and the jail. Symbolic links cannot lead out of jail, but they can lead from the outside in. And hard links do cross the chroot boundary. If the live system depends on a jailed file, and that jailed file can be modified in a malicious way, it may be possible to subvert the jail.

  • If networking utilities are available in the jail, it may be possible to attack the full system via the network. In this respect, it’s similar to a remote network attack, but this one has the added advantage of using localhost: many systems grant a bit more access via localhost than they do to external visitors.

Typically, you’ll want to jail an entire application. In fact, some applications are designed with this in mind. The BIND nameserver, for instance, has built-in support for running inside a jail. In the case of BIND, the jail greatly limits damage should a vulnerability be exploited.

But even if an application isn’t designed for jailing, it’s quite possible to “force fit” it. Some questions to consider:

  • Can the application run as a non-root user? Running as root in a jail is not safe, and must be avoided at nearly all costs. If the application requires root privileges only at startup (say, to bind to a reserved network port), it may be possible to retrofit the application with some source-code changes. However, if the application requires perennial root access, it may not be worth the trouble to jail it.

  • Does the application have a limited “footprint?” An application that requires large numbers of support files (say, the traditional UNIX/Linux utilities used by common shell scripts) might require too much work to jail. Complexity is an enemy of security, and a huge jail environment may be difficult to secure reliably.

  • Does the application require access to the larger filesystem? A jailed application can’t reach outside the jail, so an application that needs items in the full filesystem is a poor candidate for a “prisoner.” For instance, there is no reliable or safe way to reach the full system’s /etc/passwd file from a jail.

  • Can jail support be tested? Repeatedly running strace on an application can be quite tedious, since each run might only reveal one or two more files that must be jailed. To avoid surprises in production mode, you have to be sure that all code paths have been tested properly — lest a critical but not-previously-discovered file turns up missing. The more complex the application, the more unlikely it is that you can test it properly.

On the other hand, some applications are easily jailed (sometimes with a bit of support code). For our real world example we’ll use the Perforce SCM (Source Code Management) daemon p4d. p4d passes the above tests easily, and the application is simple enough to allow us to focus on jail issues, not “application behavior” issues.

Jailing with a Wrapper

Before jailing our application, we must know a little bit about it. The Perforce daemon is a network-based server that listens on TCP port 1666, and operates out of a directory that contains the source-code “depots.” p4d requires an ASCII “license” file (that describes what features you’ve purchased) and very little else. p4d is an ideal application to jail because it’s so simple.

This application will live in /chroot/perforce. /chroot is where jailed applications typically live (as opposed to /home/jail, which was just for practice). Before considering the “housekeeping” files required for the jail, the “obvious” files required are the /chroot/perforce directory, the /chroot/perforce/tmp directory (you always need a temporary directory), the /chroot/perforce/bin/p4d executable (the Perforce daemon itself), and /chroot/perforce/depot/license (the Perforce text license file).

We’ll be adding to this list as we go along, but to make it repeatable we’ll create our jail and set permissions in a shell script. That puts it in source code.

A key question arises, though: how exactly will we run p4d in jail as a non-root user? To do this, we’ve written a program — runchroot — in C that runs initially as the superuser, changes to and sets the chroot jail directory, then “gives up” root privileges and runs as the target user. It’s written generally so it can be reused for any kind of jail, usually without source code changes (you can download the source code for runchroot from http://www.linux-mag.com/downloads/2002-12/jail).

The wrapper requires that the target application user be defined in the real password file, and we’ve chosen the user name perforce. This user is added with the traditional system administration tools, and the home directory should be set to the top of the jail: /chroot/perforce. The user ID, group ID, and home directory are all consulted by runchroot.

Before attempting to run the application in jail, let’s do a bit of the routine setup as shown in the practice jail. Add the dev/null and dev/zero devices, the strace program, and install the necessary shared libraries as reported by ldd. p4d also requires a simple license file. We can test the server as root with:

# cd /chroot/perforce
# chroot /chroot/perforce strace p4d -r /depots

As before, a bit of detective work reveals the a few additional libraries are required. In practice, you’ll have to do a fair amount of tracing to exercise the target application as much as possible.

The end-result should be a make-jail script that can be used at any time to create a new jail. Listing One is the result from one system.

Listing One: A sample make-jail script

mkdir -p /chroot/perforce
cd /chroot/perforce
mkdir tmp bin dev lib etc depots
mknod dev/null c 1 3 # varies per platform
mknod dev/zero c 1 5 # varies per platform
cp /chroot/perforce.license depots/license # Perforce specific
cp /usr/local/bin/p4d bin/ # Perforce specific
cp /usr/bin/strace bin/
cp /lib/ld-linux.so.2 lib/ # varies per platform
cp /lib/libNoVersion.so.1 lib/ # varies per platform
cp /lib/libc.so.6 lib/ # varies per platform
cp /lib/libnsl.so.1 lib/ # varies per platform
cp /lib/libnss_dns.so.2 lib/ # varies per platform
cp /lib/libnss_files.so.2 lib/ # varies per platform
cp /etc/resolv.conf etc/
cp /etc/host.conf etc/
cp /etc/nsswitch.conf etc/
cp /etc/hosts etc/
cp /etc/localtime etc/

Listing One creates a new jail, but it doesn’t do anything with permissions or file ownership. Let’s create a second, “set the permissions” script that’s also handy to “clean up” a jail after manual adjustments have been made. The “set the permissions” script is shown in Listing Two.

Listing Two: A script to set permissions in the jail

cd /chroot/perforce
chown -R root.perforce . # root owns everything
chmod -R a= . # nobody can read anything
chmod ug=x .
chmod ug=rwx tmp/
chmod ug=rw tmp/* 2>/dev/null # directory might be empty
chmod ug=x dev/
chmod ug=rwx dev/null dev/zero
chmod ug=x bin/
chmod ug=x bin/strace
chmod ug=x bin/p4d
chmod ug=r etc/*
chmod ug=x etc/ # directory
chmod ug=rx lib/* # shared libraries
chmod ug=x lib/ # shared library directory itself
chmod -R ug+rw depots
find depots -type d -print | xargs chmod ug+x
chmod ug=r depots/license

This script is very aggressive: it makes root own everything, and starts with all files being unreadable by all. Permissions are then gradually opened up to permit just the bare minimum required by the runtime daemon (and excludes “regular” user access via the full filesystem). Starting with aggressively tight permissions tends to find problems right away, which in turn, leads more quickly to an optimal configuration.

The final step in setting up the jail is performing the chroot itself correctly. For the practice jail we used the command-line chroot program, but it only works for root: it’s not possible to start a program as a non-root user directly this way, so it’s time to bring in the wrapper program.

To get the wrapper parameters right, we create two more scripts that launch the jailed application. The first is the “regular” way to launch it for production, and the second runs p4d with tracing — this lets us track down the resources required by the program.

# cat /chroot/perforce.start
exec /chroot/bin/runchroot -u perforce — p4d -d -r /depots
# cat /chroot/perforce.trace
exec /chroot/bin/runchroot -u perforce — strace -o /tmp/trace.out p4d -r /depots
# chmod +x /chroot/perforce.start /chroot/perforce.trace

Now, simply running /chroot/perforce.start as root launches the application, and this can be done as part of the system boot-time scripts.

Minding the Jail

As with most security measures, it’s not enough to set up and forget: even a jail with good locks and bars still needs guards around to keep an eye on things. The only reason for a jail is added security, and not surprisingly, there are a few rules that should guide the construction.

  • Run the jail as a non-root user. This is the single most important rule. Root users in a chroot environment can nearly always break out of jail, especially via a buffer overflow in a networked application. No amount of careful design can forestall this in every case, so it’s crucial to find a way to run the application as a non-root user. It’s common for an application to require root privileges for a time, usually at startup, to bind to a low-numbered TCP or UDP network port or to access a root-only configuration file. In this circumstance, the application should launch as root, perform the privileged operations, then “give up” root privileges.

  • Tighten up permissions ruthlessly. In most cases, files in the jail won’t need to be modified by the jailed application. Those files should be owned by root (not the special jail user) and set with the most limited permissions possible. Though removing write permission alone seems obvious, if the file is owned by the jailed user, an intruder can simply chmod the file to allow writes and modify the file. Instead, if the file is owned by root, this can’t happen. Executables should be “execute only,” and directories should generally be unwritable as well (which prevents renaming, deleting, or creating of new files). For file or directory operations that must allow writes modicications can usually be made via the group permissions bits rather than by owner. This prevents the application from changing other attributes of the files.

  • Create a set-permission script. “Tuning” the permissions in any jail is tedious, and it’s very easy to forget or misplace a step. Omissions and mistakes make it difficult to recreate a jail. A README file is a good start, but it’s far too easy for the instructions in a text file to grow out of synch with the actual procedures required. The only hope to get this right is to embody the permissions policy in source code via a shell script. The Perforce example is illustrative.

  • Put as little as possible inside the jail. The more “stuff” that’s in the jail, the more complicated and insecure it is. Only put items in the jail hierarchy if they’re required.

  • Watch the log files. Most jailed applications still support logging, either through syslog (with some setup required) or into files. It’s important to watch the logs for evidence of intrusion or failures. Especially for newer applications, it’s common to “trip across” required missing files that weren’t discovered during initial construction. When adding the new files, be sure to update the construction and permissions scripts.

  • Keep the application patched. Though the jail may keep the bad guys out of the larger system, breaking the application itself is still possible, and that can have negative consequences (say, getting the data from a jailed MySQL database). Setting up a jail doesn’t mean you should unsubscribe from BugTraq.

Safe In Jail

A chroot jail protects against unforeseen vulnerabilities, and it’s saved many a system from getting cracked by that new vulnerability the administrator hasn’t gotten around to patching.

Hopefully, this second-level of security, a kind of “backstop”, will be adopted by more administrators and developers.

Now if we can just close down all those open mail relays.

Steve Friedl is an independent consultant who has used UNIX for more than 20 years, and he can usually be found working at home in his pajamas. He gets his mail at steve@unixwiz.net.

Comments are closed.