The Brawn Behind the Brains

If you were to install nothing but the Linux kernel on your hard disk, it would not do very much. A surprisingly large part of Linux's power and flexibility come not from the core of the operating system or from the applications that come with it, but from daemons. These special background programs are typically run without user intervention, and provide everything from printing to file services to access to Web pages.

If you were to install nothing but the Linux kernel on your hard disk, it would not do very much. A surprisingly large part of Linux’s power and flexibility come not from the core of the operating system or from the applications that come with it,
but from daemons. These special background programs are typically run without user intervention, and provide everything from printing to file services to access to Web pages.

Because daemons are separate programs, and not part of the operating system proper, they are usually loaded separately from the kernel. In most cases, simply installing a particular package is sufficient to activate a daemon, if one is needed.

A number of daemons are started up automatically by a special script during different points in your operating system’s life. How this script works and how you can change the behavior of your machine’s daemons is what we’re going to talk about in this column.

Letting the Daemons Loose

Once the kernel is loaded, one of the last things it does is start a special program (or “process”) named init. One of the init process’ jobs is to manage when the various daemons start and stop. The appropriate time to start and stop a daemon depends on a number of things. For one thing, your system may need to stop some daemons to perform certain administrative tasks. Many systems disallow network connections or even disable NFS when doing backups, for example.

Although you can stop these daemons yourself, the system provides a mechanism to do this automatically. To understand how this mechanism works, we first need to talk about a concept known as “run levels” (which are also called “run states”). A run level is a set of instructions, executed by the system, that control the activation or de-activation of particular system services or daemons.

Most users (and many administrators, for that matter) are familiar with only one run level. This is the run level in which the systems is performing all of its normal functions. Users can
login, submit print jobs, access Web pages, and do everything else one would expect. This run level is commonly referred to as multi-user mode. In contrast, the maintenance or single-user mode run level is normally recommended for administrative tasks.

Each run level is referred to by its number. When the system is shut down (as opposed to running normally but with nothing to do), it is at run level 0. Single-user mode is run level 1. Most Linux systems automatically boot into level 3. Run level 2 is very similar to run level 3, though it has fewer services. In fact, on some systems (SCO UNIX, for example), run level 2 is the standard multi-user mode. (For more details on the run levels, type the command man init to take a look at the init(8) man-page).

Like many other aspects of the system, init has its own configuration file: /etc/inittab (see Listing One). This file contains the init table (inittab), which tells init what to do and when to do it. Use the command maninittab to read the man page for this file, which includes several interesting options not covered in this article.

Listing One: Excerpts From the /etc/inittab File

 # Default runlevel.
# System initialization.
si::sysinit:/etc/rc.d/rc.modules default
# What to do in single-user mode.
~1:S:wait:/etc/rc.d/rc 1
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

Each activity init performs is represented by a single line entry in the inittab. These entries looks something like this:

sbin/getty tty1
VC linux

The entry is made up of four fields, each separated by a colon. The first field (1, in this case) contains a unique identifier or “label” for that entry, which enables init to keep track of each entry it has run. The second field (here, 12345) tells init during which run levels this particular entry will run. The third field (respawn)
is the “action” field, which tells init how to behave. For example, some inittab entries are only processed when the system boots.

However, some processes started at boottime must be perpetually re-started whenever that process is killed (for example, a getty process that monitors a terminal and waits for someone to log in). Well, this is what the respawn action does — it tells init to restart a particular process every time that process is killed. The last field (/sbin/getty tty1 VC linux) contains the name of the program being started. This field also often includes a number of options for that program.

If you look at your system’s
inittab, you may notice something peculiar. More than likely, you are not going to find any entries for individual daemons. The reason is, quite simply, that the daemons are not started through the inittab, but rather through scripts which are started from the inittab. These scripts are called in the entries labeled l0 through l6, for run levels (“l”) 0 through 6.

In Listing One, the wait action means that inittab will wait until the rc script is terminated before it continues on and processes other entries for this run level. This also means that the entry in question will be processed only one time as the system enters that particular level.

The key to all of this is the program which is run for each run level. In Listing One, it is the shell script rc that is given the appropriate run level as an argument. This script is often called the “run level master script,” as it is responsible for loading all of the other init scripts. Where this script is hidden and what it is called changes depending on your distribution. Under SuSE it lies in /sbin/init.d/. Under Caldera and Red Hat the script resides under /etc/rc.d. Note that SuSE also has an /etc/rc.d directory, which is actually a symbolic link to /sbin/init.d.

The actual code in the script also changes depending on your distribution, but all versions basically do the same thing. They all start other scripts which finally start the daemons we have been talking about.

Finding The Right Level

As we mentioned previously, an argument passed to the rc script tells the system which run level it should be at, and what daemons to run. At least that’s part of it. In addition, the system needs a list of which scripts should be started for a given run level. These scripts are separated into different directories, one for each run level.

If you look in the /sbin/init.d or /etc/rc.d directory, you’ll see a number of subdirectories of the form rc#.d, where # is a particular run level. For example, the subdirectory rc3.d refers to run level 3. The subdirectories contain symbolic links to the actual scripts. The primary reason for this is that a script can be started in more than one run level. So if the files were copies instead of symbolic links, any change would have to be copied into every single file — which can really be a pain. Symbolic links rather than hard links are used because they allow us to point to files on other file systems. With SuSE, the /sbin/init.d directory is also where the real scripts reside. On the Caldera and Red Hat distributions, the scripts reside under /etc/rc.d/init.d.

The filenames are a little different from what you may be used to, but once you understand how to interpret them, they’re really very straightforward. As you might guess, the link ending in apache points to the script which starts the Apache Web server. This link points to the script /sbin/init.d/apache. Other links have similarly recognizable endings. However, you’ll see there are actually two files with each type of ending. The really odd thing is that both of these links point to the exact same file. For example, you may find a file called S09samba and another one called S91samba, both of which obviously have something to do with samba. So, what’s the deal?

Each of these links starts with either the letter S or the letter K. Those which begin with the letter S are used to start the particular service and those which begin with the letter K are used to stop (or kill) that same service. That leaves us with the numbers. These are used to define the order in which the scripts are to be run. When the files are listed, they automatically appear in numerical order so the system can run them in the correct order. This helps to prevent things like starting your Web server before you start your network.

The same logic applies when the system shuts down. K20apache is used to shut down the Apache server and K40network is used to shut down the network. As in the first case, the network is not shutdown until after Apache has been killed.

It is interesting to note that this system could work even if the symlink’s name consisted of only S or K and the appropriate number (K40 and K20, for example). But the names (networkand apache, in our example) are added to avoid unnecessary stops and starts to various services.

An example will make this clear. In the directory rc3.d (for run level 3), there are links used to both start and stop the network. Run level 1 on the other hand, does not use the network and therefore does not have those links. Thus the network will always be started when moving from run level 1 to run level 3. Conversely, the network will always be stopped when moving from run level 3 to run level 1, due to run level 1′s lack of a network “start” link.

On the other hand, there are also links to both start and stop the network in rc.2 (for run level 2). Therefore, when leaving either run level 2 or 3 and moving to the other, the network is not stopped as there is a start link for the network in both the new run level and the previous run level. However, when moving from either run level 2 or 3 to run level 1, for example, the network is stopped.

As I mentioned previously, the start and stop links both point to the same file. The script needs some way of knowing whether it should start or stop the service. This is done by passing an argument to the script: start to start the service and stop to stop the service (simple, huh?). Each script, performs different activities based on what the argument was.

Note that for many scripts, you can pass more than simple start and stop arguments. For example, one common argument is restart. Many scripts will also accept the status argument, which is used to deliver status information about a particular service. These arguments are typically passed by the system administrator and not by the rc script.

More Than Meets the Eye

On many systems, the run level scripts we have talked about so far are not the only scripts that are started when the system boots. Remember that init reads the inittab to find out what to do, so there are any number of things that “could” be started through the inittab. Even so, if you are used to a different flavor of UNIX, like Solaris or Digital UNIX, the inittab on a lot of Linux systems looks pretty barren.

One type of script that is often run from the inittab deals with system initialization. For example, the boot script, which is found directly in /sbin/init.d. The entry in the inittab might look like this:


The run level this script runs in is “I,” which is not a traditional run level, but used by some distributions (SuSE, for example) to indicate system initialization. However, because the action is bootwait, the run-level field is actually ignored. bootwait means that this entry will be processed while the system boots, and init will wait until the command or script has completed before proceeding.

In this case, the script is /sbin/init.d/boot, which performs basic system initialization such as starting the bdflush daemon (which writes dirty buffers to the disk), checking the filesystems (with fsck), mounting filesystems, starting the kernel module daemon (kerneld), and many other things.

Other versions of Linux may have different entries in the inittab, which combine to do the same work as the /sbin/init.d/boot script under SuSE Linux.

Note that this is one place where there is also some divergence amongst distributions. For example, both Red Hat and Caldera have a halt script (under etc/rc.d/init.d), but no boot script. The boot script functions are performed elsewhere.

There are also scripts that carry out procedures when the system is brought down. This occurs when the system is set to run level 0 (halt) or run level 6 (restart). In general, these reverse the procedures in the boot script, such as stopping the kernel module daemon and unmounting filesystems.

SuSE also makes use of a system configuration file /etc/rc.config. This file contains a large number of variables, which are used to configure the various services. Reading this file and setting the variables is one of the first things done by the script /sbin/init.d/rc. The counterpart to this file on Caldera is the directory /etc/syconfig/daemons. Instead of a single configuration file, you will find separate files for a number of different daemons. Red Hat has neither, but
it does have the file /etc/rc.d/init.d/functions, which contains a number of functions used by the rc scripts.

Creating Your Own init Scripts

You may eventually find that your particular distribution’s scripts are not sufficient and you need to add your own. I have created my own init scripts on a number of systems where I have needed to add my own system services. The best way of doing this is to simply follow the conventions used by your distribution and put the files in the right directories and create the right links.

SuSE has realized the need for creating your own init scripts, and so has provided a template for you. This is the file /sbin/init.d/skeleton and, as its name implies, is a “skeleton” init script. In its default state, this is a runable but useless script, since it doesn’t actually start any daemons. Instead, you simply uncomment the lines you need, change the name of the daemon or service, and you are ready to run.

James Mohr is the author of books on Linux, SCO UNIX, Web site administration, and UNIX-Windows integration. He works in Coburg, Germany for a systems integrator, and can be reached at jimmo@blitz.de.

Comments are closed.