In the Beginning — Part II

Last month, we looked at the events that transpire when you boot up a Linux system, from the kernel being loaded to the init process getting started. This month we turn our attention to actions that follow init's startup -- the actions performed by the boot scripts that actually take care of all the tasks necessary to make the system ready for users.

Last month, we looked at the events that transpire when you boot up a Linux system, from the kernel being loaded to the init process getting started. This month we turn our attention to actions that follow init’s startup — the actions performed by the boot scripts that actually take care of all the tasks necessary to make the system ready for users.

We will begin with the system’s initial boot script (as would be specified in the bootwaitinittab entry), called rc.sysinit (Red Hat) or boot (SuSE). The script is responsible for the fundamental and essential tasks that comprise a successful boot process; it must execute successfully before the system can accept any user logins. The tasks performed by this script include (in a typical order):

  • Mounting the /proc filesystem
  • Starting the kernel daemon, kerneld, which dynamically loads kernel modules
  • Initializing RAID devices and LVM-related entities
  • Running fsck on any filesystems to be checked
  • Loading required kernel modules
  • Mounting all filesystems
  • Initializing basic network devices, which would include support for remote booting
  • Preparing several other fundamental devices and facilities
  • Performing various cleanup activities; these would include removing old lock files, some temporary files, lingering configuration items that are no longer needed (e.g., /etc/ nologin), and performing other similar tasks

The SuSE version of this file contains a hook that enables a system administrator to add items to the process. The relevant commands are to be placed in the file boot.local and will be executed near the end of the main initial boot script.

The rc Script

Once the initial boot script has finished, the boot process generally continues by executing the main boot script, rc, which is called with the new run level as its argument. The script functions as the overseer for the rest of the boot process; as such, it performs few actions directly itself, but rather relies on a series of other, specialized scripts to do each job that needs to be done. An idealized and simplified version of the rc script (derived from one on a Red Hat system) is given in Listing One.

Listing One: An Idealized rc Script

#! /bin/bash
. /etc/rc.d/init.d/functions

[ "$1" != " ] && runlevel=”$1″

if [ -d /etc/rc.d/rc$runlevel.d ]; then

# First, run the KILL scripts.
for i in /etc/rc.d/rc$runlevel.d/K*; do

# Check if the script is there.
[ ! -f $i ] && continue

# Check if the subsystem is already up.
[ ! -f /var/lock/subsys/$subsys ] && \
[ ! -f /var/lock/subsys/${subsys}.init ] && continue

# Bring the subsystem down.
$i stop


# Now run the START scripts.
for i in /etc/rc.d/rc$runlevel.d/S*; do

# Check if the script is there and subsystem is already up.

# Bring the subsystem up.
$i start



The script begins by loading some functions from an external file and then sets the variable runlevel to the value of its first argument. The main part of the script consists of the two for loops enclosed within the outer if statement. The latter determines if a subdirectory named rcn.d exists in the /etc/rc.d directory, where n is the new run level. If rcn.d exists, the script goes on to execute the files it finds within that run level’s subdirectory.

These are the files that will perform the actual tasks required to get the system up and running: initializing any remaining devices (e.g., serial/ terminal lines, printers), setting up full networking, starting all the various system daemon processes for different subsystems (for example, syslog, cron, etc.), and the like.

We have yet to say much about the locations of the various system boot scripts. Traditionally, the inittab file is stored in /etc, and current Linux systems continue to follow this convention. On most Unix systems the main boot scripts — rc.sysinit, boot, and rc — reside in either /etc or /etc/ rc.d, and Linux systems generally use the latter location. This directory also holds a series of subdirectories named rcn.d for each of the defined run levels (as we have just seen).

Guru/ rc_dirs OUTLINES
Figure One: The traditional Unix boot scripts directory structure.

The files within each rcn.d subdirectory have names beginning with either “S” or “K” followed by a two-digit number (we will discuss the significance of this in a moment), to which a short subsystem name is appended. However, each file is actually a link to a file in the subdirectory /etc/rc.d/ init.d which has the same subsystem name. This directory structure is illustrated in Figure One.

In our example, the file named at, which lives in the init.d subdirectory, also exists (as a hard link) in the rc0.d subdirectory (named K20at) and in the rc3.d subdirectory (named S20at). Other files in the init.d subdirectory are similarly linked to files in these two rcn.d subdirectories.

In contrast, on recent SuSE Linux systems, this usual directory structure has been altered (as is also the case for some other Unix operating systems). The main boot scripts have been moved to a subdirectory of /sbin, in this case, /sbin/init.d. The rcn.d subdirectories also reside in this location, and the files within them are still links to the corresponding files in an init.d subdirectory at their same location within the directory tree. However, the subdirectory /sbin/ init.d/init.d does not exist but is rather a link to /sbin/init.d itself, and the subsystem-specific scripts actually reside there as well. Currently, /etc/rc.d is itself a link to the new location /sbin/init.d.

Keeping all of this in mind, we are now positioned to discuss each of the two loops that make up the body of the example rc script. Each for loop is constructed to run over a list of files found in the appropriate run level-specific subdirectory and specified with a wildcard in the for command. For example, the first loop runs over all the files in the appropriate directory whose names began with a “K.” These files are used to shut down or stop processes and subsystems and are called “kill” files. The second loop runs over the “S” files, which are used to initiate and “start” processes and subsystem.

Starting and Killing Subsystems

At this point, it would be helpful to look at a concrete example of a directory listing from one of the subdirectories. Example One contains such a listing from a Red Hat Linux system.

Example One: A Sample rcn.d Listing

$ ls /etc/rc.d/rc3.d
K08autofs K20bootparamd K30mcserv K60atd S30syslog S60nfs
K09keytable K20rusersd K30sendmail K60mars-nwe S40crond S85sound
K10named K20rwalld K30ypbind K75gated S40portmap S91smb
K10pnserver K20rwhod K33yppasswd S01kerneld S45pcmcia S99local
K15gpm K25innd K35dhcpd S10network S50inet
K15httpd K25news K35ypserv S15nfsfs S55routed
K15postgresql K28amd K40snmpd S20random S60lpd

The files within the subdirectory have the naming format that we have previously described. Note that the two-character number following the initial key letter creates an explicit sequence among the files. When the wildcard “K*” is expanded in a subdirectory, for example, the filenames of matching files will be returned in numerical order, and the same order will be consequently followed as they are executed. When we look at the sequence of files, “K” files followed by “S” files, we can see the progress of the second phase of the boot process in detail and follow along as the subsystems are initialized one by one.

Assuming that the for command within the script finds some files, the loop first checks to see that the file exists. Then it extracts the subsystem name from the end of the filename (using a bash variable construct) and examines the /var/lock/subsys subdirectory for lock files related to that subsystem; this step is designed to avoid trying to stop the subsystem that is not in fact running. Finally, if these tests have all been passed, the loop runs the script, passing it the parameter stop.

The second loop is analogously constructed and examines and runs the “S” files in the same subdirectory, in this case passing the parameter start to the script. Note that, in order to save space, we have omitted the commands for the various tests in the second loop in Listing One.

To summarize, on a Red Hat system, the files in the corresponding rcn.d subdirectory are all executed whenever a new run level is entered. Specifically, the “K” files are executed in order, followed by the “S” files in order.

Things are somewhat different on recent SuSE systems. The rc script that is provided with this Linux distribution first runs the “K” files in the rcn.d subdirectory corresponding to the previous run level and then the “S” files in the subdirectory corresponding to the new, target run level. SuSE systems also do not use subsystem-specific lock files in /var/lock/subsys.

Since all of the scripts within a run-level subdirectory are links to the real script in the init.d subdirectory, it follows that the same script is run, regardless of whether it is executed as a “K” file or as an “S” file. Listing Two contains a sample script from a Red Hat Linux system; this one is concerned with the SAMBA subsystem.

Listing Two: SAMBA Script on Red Hat


# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

# See how we were called.
case “$1″ in

echo -n “Starting SMB services: ”
daemon smbd -D
daemon nmbd -D
touch /var/lock/subsys/smb

echo-n “Shutting down SMB services:”
killproc smbd
killproc nmbd
rm -f /var/lock/subsys/smb
echo ”

echo “Usage: smb {start|stop}”
exit 1


This sample script again began by loading some library functions and goes on to run a networking-related configuration file stored in the /etc/ sysconfig subdirectory. The main part of the script consists of the case statement, which switches on the value of the scripts first argument, allowing the script to behave differently depending on whether that argument is start or stop (corresponding to being executed as an “S” file or as a “K” file, respectively).

In the first case, a message is printed to the console, and the SAMBA-related server processes are started via the daemon command. Finally, the lock file corresponding to the script is placed into the subsystem-specific subdirectory of /var/lock.

When called with the parameter stop, the script again sends a message to the console indicating what it is doing, terminates the Samba server processes (via the killproc command), and then removes the lock file.

Any parameter other than these two will cause a usage message to be displayed, after which the script will simply terminate.

All of the boot scripts on Linux systems (and on Unix systems in general) follow this general format. There are slight variations among the various scripts and across Linux distributions. Listing Three contains a brief excerpt from the SAMBA script (named smb) on a SuSE system.

Listing Three: SAMBA Script on SuSE

. /etc/rc.config

test “$START_SMB” = “yes” || exit 0


case “$1″ in

echo -n “Starting SMB services:”
startproc /usr/sbin/nmbd -D || return=$rc_failed
startproc /usr/sbin/smbd -D || return=$rc_failed
echo -e “$return”

Rather than loading some shell functions at the beginning, the script reads the contents of the /etc/rc.config configuration file. The functions of this file are to specify which subsystems should and should not be enabled on the system and to set various characteristics for some of them. The second line of Listing Three illustrates this concept. It tests the value of the variable START_SMB and exits if that variable is set to anything other than yes.

Guru (Boot)
Figure Two: SuSE boot messages like these are typically color-coded.

The start section of the case statement looks very, very similar to the one from the Red Hat system (shown in Listing Two), although a different program is used to start the SAMBA daemons. These commands also illustrate the method used to obtain SuSE’s distinctive color-coded boot messages. The second echo command displays the value of the variable return, which is set to one of two strings defined in rc.config, depending on whether or not the script is successful. The relevant lines in the configuration file then format the word “done” or “failed” sandwiched between escape sequences that alter the terminal display color. The result is shown in Figure Two.

Customizing the Boot Process

The sequence the boot scripts are run in is obviously very important. Dependencies among the various subsystems of the operating system must be satisfied in order for a fully functional system to be the end result; if a daemon is started before one that it depends on is started, your system will probably die and not be available unless it is restarted manually. For this reason, it is best not to alter the ordering of these files until you are very familiar with the entire boot process and each individual subsystem.

Keeping this in mind, there are nevertheless several safe ways of customizing the boot process for the needs of a particular computer system or organization. First of all, be sure to take advantage of whatever customization facilities are provided with your Linux distribution. For example, on SuSE systems you can modify the /etc/rc. config file settings to enable or disable various subsystems. In addition, many of the standard boot scripts have configuration files designed to customize their functioning.

For minor additions to the boot process you can use the built-in customization facility within the main boot script itself. Many versions of the rc script execute another script, conventionally named rc.local and stored in the same location as rc itself, just before exiting. Additional commands that you require can be added to this file. If your version of the rc script does not have such a hook, you can add one by placing a command like the following at the end of the script:

. /etc/rc.d/rc.local

You will then need to create rc.local. You’ll also need to similarly modify new versions of the rc script that are installed with any system upgrades.

Guru (ksysv)
Figure Three: The ksysv utility can be used for modifying the boot process.

For more major changes, you may need to activate/deactivate, re-order, or add to the sequence of boot scripts in the run level subdirectories. This can be done manually by adding or removing properly named hard links to files within init.d from the appropriate subdirectory (and possibly by adding new scripts to init.d).

Alternatively, there are GUI tools which make this process more convenient. The best known of these is sysvinit; a variant created for the KDE graphical environment, ksysv, is illustrated in Figure Three (this tool is also accessible from the KDE menu).

The utility lists the contents of the init.d subdirectory in the leftmost pane, and the contents of the various rcn.d subdirectories are shown in the series of panes moving across the window to the right; “S” files and “K” files appear in separate lists. Items can often be reordered within a subdirectory by simply dragging them to the new position. Similarly, you can add a new script to any run level subdirectory by dragging the desired item from the init.d list to the desired location within the appropriate window. This process is illustrated in Figure Four, where we add (activate) a script to run-level 3 that starts the mysql facility.

Guru (Added)
Figure Four: Adding a boot script to run level 2.
Guru (Props)
Figure Five: You can also fine tune the boot script sequencing.

When you drop a file into a run- level subdirectory’s list, it is automatically assigned the appropriate filename; the correct initial letter is chosen, and a number falling in sequence with those above and below the specified location is computed. Sometimes, however, you will need to fine-tune the utility’s default numbering. This is accomplished by right-clicking on the icon whose number you wish to modify and changing it manually in the resulting dialog that displays all the items properties (illustrated in Figure Five).

Removing a script from one of the run-level subdirectories is equally easy. You simply click and drag the item from the list to the utility’s trashcan, located in the bottom left portion of the window.

Accept No Substitute

That’s it for our Linux boot sequence overview. The addition of the GUI tools we looked at should make your life easier if you need to dig in and mess around with your system’s boot sequence. Still, even with the addition of easy-to-use graphical interfaces, there is just no substitute for understanding what’s going on under the hood.

Æleen Frisch is the author of Essential System Administration. She can be reached at aefrisch@lorentzian.com.

Comments are closed.