dcsimg

A Tour of the Linux File System

Take a tour of the Linux filesystem. Drive down /bin Street, cruise over to /usr Lane, and buy some souvenirs over in /dev Drive.

I just purchased a GPS navigation device, and after only a handful of uses, I am hooked. Getting from Point A to Point B is now a snap. No more MapQuest. No more guessing which way is east. No more pit stops in the sticks to ask for clarifications. I simply jump in my car, specify my destination, and follow the voice prompts. Why, the GPS makes me look like a local, giving new meaning to the Buckaroo Banzai maxim, “Wherever you go, there you are.”

Let’s indulge my wanderlust this month and ramble around the Unix filesystem. From /bin to /var, there are lots of interesting sights to see — some well-trodden and others obscure — and shortly, you’ll know your way around like the locals.

What’s in a file name?

The files on a Unix machine are organized in a hierarchy. The very top of the hierarchy is / — commonly referred to as “slash” or “the root directory”.

If you change directory to / and run ls, you’ll see a number of subdirectories, with cryptic names like “etc”, “bin”, “var”, “home”, and “tmp”. Although Unix now supports long file names, most of the monikers of these top-level directories hark back some thirty years to Unix’s origin. Similarly, by the same long-standing conventions, each directory contained in / serves a special purpose.

  • /bin is but one of many directories that contain applications and utilities. However, /bin typically contains utilities that are essential to system operation. Hence, the shells, file manipulation commands such as cp and chmod, compression and decompression, and diagnostics are found in /bin.

    /sbin also contains utilities crucial to system operation and maintenance. However, the programs found in /sbin can be executed only by the superuser — hence “superuser-bin” or /sbin.

  • /dev incorporates all of the hardware installed on your system, including terminals and USB devices (physically connected to the computer), pseudo-terminals (used to interact with X terminal windows), and hard drives, among others.
  • /etc (often pronounced “etsee”) is dedicated to system configuration. /etc contains configuration files for the system daemons, startup scripts, system parameters, and more.
  • /home contains all of the users’ home directories. For instance, if your login name is joe, the directory /home/joe acts as your personal file repository.
  • /lib is the coffer for essential system libraries. In modern Unix, system libraries are typically shared, meaning the libraries are not compiled into each binary (which would waste space, in the least), but are loaded on demand when needed and shared by many applications at once. Hence, core applications and utilities installed with Unix require the libraries in /lib to run, and you need at least a small handful of the libraries to create new executables from source code. All files here are vital, and the corruption or removal (whether intentional or accidental) of even one file can render a system useless.
  • /mnt, short for “mount”, is the standard location to mount hard drive partitions and other devices. If you want to see what devices are currently mounted and accessible, simply run the command mount.
  • /tmp, or “temporary” is the system-wide scratchpad. Your Web server may stash session data files here, and other utilities use the space in /tmp for caching intermediate results. Files in /tmp are considered to be disposable. Indeed, your system administrator probably deletes all files older than a certain expiry every evening.
  • /usr is the umbrella for a great number of files. End-user applications, such as editors, games, and interfaces to system features are here, as is the library of man pages, and more. Chances are that if the file is useful, but not mandatory for system operation, it’ll be found in /usr.
  • /var — short for “variable” — is the repository for files that typically grow in size over time. Mailboxes, log files, printer queues and databases can be found in /var. It’s commonplace for Web sites to be kept in /var as well, since a Web site tends to amass data preternaturally over time.

The directory names above are the most common, although some flavors of Unix diverge slightly. (For example, on Mac OS X, which is based on FreeBSD, the directory that contains the users’ home directories is named /Users rather than /home.)

Keeping Tradition

In fact, the names “etc”, “bin”, “lib”, and “man” are so entrenched in Unix culture that it is traditional to label directories of like purpose found elsewhere on the machine with the same name. For example, if you look at an expert’s home directory, you’ll likely find bin and lib to store personal applications and scripts and personal libraries, respectively.

Mimicking canon, /usr/local frequently has “etc”, “bin”, “lib”, and “man”. Historically, /usr/local has been used to store applications and data that originate or are germane solely to your site. /usr/local/binstores locally-added, new programs, and locally-modified versions of standard system utilities. For example, your system administrator may offer the latest and greatest version of Perl in /usr/local/bin/perl, keeping /usr/bin/perl constant, both for reference and because other core utilities may depend on it. /usr/local/lib complements /usr/local/bin.

/usr/local may even be an entirely separate partition (even a partition mounted via the Network File System from a network-attached storage device), which makes system restores and resurrections simpler — if something happens to a system, the administrator can overwrite the operating system’s files without worrying about destroying local data.

Even packages mirror the root directory. Consider MySQL: if configured with the option --prefix=/usr/local/mysql, it creates its own “root” directory in /usr/local named /usr/local/mysql, and creates subdirectories /usr/local/mysql/bin, /usr/local/mysql/lib, and so on.

$ ls -1 /usr/local/mysql
bin/
configure*
data/
docs/
include/
lib/
man/
...

Alternatively, if you want to install MySQL’s pieces in /usr/local/bin, /usr/local/lib, et al, use --prefix=/usr/local.

Other Points of Interest

Since this is the twenty-five cent tour, let’s swing by a few other attractions.

/etc

/etc is the place to look for configuration files, which usually end with the suffix “.conf”. A larger package may have its own subdirectory to collect all of the configuration files for that package. One case in point is Apache; in particular, Apache 2.2 has reorganized its configuration files to be more modular and less monolithic. Other novelties include

  • /etc/init.d accommodates the many startup scripts that run when your system boots. If you want to cleanly restart a daemon, say, after changing its configuration, look in /etc/init.d for an eponymous script. For example, to restart the Postfix mail transport agent (MTA), you’d run:

    $ /etc/init.d/postfix restart
    

    /etc/init.d also contains scripts to drop to single-user mode, restart, shutdown, and disable logins.

/var/spool

As mentioned earlier, /var keeps files that tend to grow and shrink in size over time. Like /, /var is subdivided into subdirectories, each with its own scheme.

  • /var/spool/mail is where to find your and others’ incoming mail. Your mailbox is simply a flat (contiguous, non-indexed) file (unless your system administrator is using the “maildir” format). Incoming mail is appended to the end of the file; mail you discard is deleted from the file; and when you read a new message, the status field of the message is changed and rewritten in place. You can read and write your own mailbox, but permissions prevent you from accessing other users’ mailboxes. (It’s recommended to not edit your mailbox directly.)
  • /var/log maintains the suite of system log files, or those files that record system activity. Logs track everything from mail traffic to failed login attempts. Each daemon usually has its own log file, which makes it easy to hunt down issues when a service fails. Since system activity can be revealing, logs here are typically restricted and available to the superuser only.

If your system provides a centralized fax service, /var/spool queues those requests, too.

/usr/man

The core man pages for your Unix system are found in /usr/man. Extended collections of man pages can also be found in /usr/local/man and in a package’s man directory, such as /usr/local/mysql5/man.

Because man pages can be found in as many places as executables, the man program supports a MANPATH environment variable that works identically to PATH. To search more than one location for a specific page, define MANPATH as a series of man page directories.

MANPATH="/usr/man"
MANPATH="/usr/local/man:$MANPATH"
MANPATH="/usr/local/mysql/man:$MANPATH
MANPATH="$HOME/man:$MANPATH"
export MANPATH

Here, $HOME/man is searched first (it’s leftmost or first), followed by /usr/local/mysql/man, and so on. By the way, the first four commands above could be simplified to the statement:

MANPATH="/usr/man:/usr/local/man:\
/usr/local/mysql/man:$HOME/man"

Yet keeping the additions separate allows you to reorder the entries quickly and add new directories just as simply. Moreover, if you have a lot of paths, editing the MANPATH (or PATH) variable becomes tedious.

Include files

Include files, or header files, define constants, macros, and other structures used in the operating system or in a particular library. Rather than redefine a specific structure, you simply “include” the header file in your code (a simple form of code reuse) and code to the header file’s specifications. (man sections 2 and 3 are set aside for such specifications; try man 2 signal, for instance.)

Akin to “bin” and “lib”, “include” is a common directory name. If a package has a development kit and you’ve installed the package in its own root directory, look in the subdirectory include for the header files.

Or, if you’ve installed a package into the common /usr/local/{bin,lib,include}, then look for a package’s header files in a subdirectory of /usr/local/include named after the package. This is an exception to storing everything in a common pool — why? Header files are not uniquely named, so installing everything in one place would cause collisions, with one package overwriting another’s header files.

If you build applications from source code (something you’ll explore in-depth in an upcoming installment) and header files are in a non-standard location, you may need to add a -I option to your compiler commands. As an example, if your ImageMagick header files live in /opt/include/magick, add -I/opt/include/magick to the compiler’s switches.

Know It Like the Back of Your Hand

This concludes today’s Unix tour. You can now navigate the alleys and backstreets of Unix a little easier. If you do get lost, just say “Home, Home, Home” (don’t get tricked by Betelgeuse) — or type cd. Remember that you can also use find and locate to find most anything, including executables, libraries, and include files.

Good afternoon, ladies and gentlemen. The next tour leaves in thirty minutes.

Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62