Software Packaging with RPM

If you're running RedHat or a Linux distribution based on RedHat, chances are you've had occassion to use RPMs. RPMs and Red Hat's accompanying package management system (and other systems like it) greatly simplify the task of maintaining the software on a system. With RPMs, installs, upgrades, and even downgrades are quick and easy.

If you’re running RedHat or a Linux distribution based on RedHat, chances are you’ve had occassion to use RPMs. RPMs and Red Hat’s accompanying package management system (and other systems like it) greatly simplify the task of maintaining the software on a system. With RPMs, installs, upgrades, and even downgrades are quick and easy.

Because RPMs make installations uniform and simple, most open source projects are distributed as RPMs (as well as other forms of packages). With a little effort, you can distribute your code and binaries as RPMs as well. You need some new tools to build RPMs, but they’re relatively easy to master. This month, let’s review the basics of package management and outline the process of creating your own RPMs.

A Brief History

A software package is a collection of related files, ranging from the typical collection of binaries and man pages, to full-blown, expansive software development kits (SDKs) that includes tools, libraries, documentation, and source code.

Before the advent of formal package formats and package managers, people typically archived and distributed software in a tarball — a tar file or a compressed variant of a tar file — and installed software by extracting tarballs on their systems. While useful, the simplicity of a tarball is also its Achilles’ heel: a tarball is just an inert collection of files, and once extracted, there’s no way to match an installed file to its tarball (or to other related files) or reconstitute the tarball. A tarball doesn’t describe its prerequisties or its incompatibilities, either.

Formal package managers address these and other concerns. Solaris uses pkgadmin, HP-UX has SD-UX/depot, Debian uses apt, and Red Hat provides RPM. True software packages include metadata as well as archived files, and package managers use this metadata to inject intelligence into file handling. Metadata expresses version numbers, prerequisites, package inventory, and can also include install and uninstall scripts that execute at package installation and removal, respectively. All in all, package managers provide coordination, tracking all of the pieces in a package.

It’s a Big Deal

If you write software — either free or commercial — chances are you stand to benefit from bundling your product in a native, distribution-specific package. Raw source code is of use only to those with the resources to build from scratch, and tarballs of binaries slip under the radar of the system’s package manager.

System administrators can also benefit from creating custom RPMs. Most shops accumulate a set of homegrown tools, and packaging those tools simplifies their distribution and maintenance. Formally packaging and versioning such tools also reinforces the notion that revisions of the tools are proper software releases, which discourages ad-hoc changes. Similarly, system administrators can create site-specific versions of pre-existing RPMs by tweaking source packages (called SRPMs), performing local builds, and distributing the new, custom packages.

Software To Go

RPM provides all of the common package-management features described above and then some:

* PROVIDES STRONG YET FLEXIBLE DEPENDENCY CHECKS. Package prerequisites may be expressed in terms of other package names or even files.

* SUPPORTS SIMULTANEOUS VERSION INSTALLS. By default, some package managers maintain only one version of a given package at a time: installing a new version of a package, say, typically overwrites a preexisting version. RPM intelligently differentiates between “install a new version” and “upgrade,” so if you plan ahead you won’t suffer from file collisions.

* SETS NO ARBITRARY LIMITS ON LENGTHS OF PACKAGE NAMES. Those of you who have created Solaris packages may recall a nine-character limit for package names, which is effectively limited to six characters if you follow Sun’s convention of using the first three characters of the name to identify the vendor or packager. RPM names have no limits.

* ALLOWS A PACKAGE TO BE REBUILT FROM SOURCE. Whereas RPMs are archives of binaries and install-related metadata, source RPMs are archives of source code and build instructions. You can unpack and change an SRPM (say, to make site-specific adjustments), then repackage and/or rebuild. The new RPM is not a hack, but a true package that is handled by the RPM system as any other.

* SUPPORTS SUBPACKAGES. You can divide your package into pieces that may be installed separately, such as distinct “client” and “server” subpackages.

While other package managers may include some of these features, RPM provides them all.

The RPM Spec File

RPM’s build model is simple: pass source code, patches (if any), and instructions about how to build and install the software through the rpmbuild command to produce binary and/or source packages. rpmbuild can be found in the rpm-build package, which you must install to build RPMs. (Building an RPM was formerly performed with the rpm command itself, but this changed as of RPM v4.1, which was released with RedHat 8.0.)

The names of the original source bundle, the names of the patches, the instructions, and a description of what files to include in the final RPM are all specified in a spec file, as shown in Listing One. The spec file controls rpmbuild in the same way that a Makefile controls make.

Listing One: A sample, prototypical spec file

%define product_namewidget
%define product_version0.41.0beta
%define rpm_release1
%define product_nameversion%{product_name}-%{product_version}
%define product_source_dir%{product_nameversion}
%define product_tarball%{product_nameversion}-src.tar.gz
%define product_man_section8
%define product_urlhttp://www.your.domain/
%define product_vendorWidgets Unlimited
%define product_licensethe very flexible Widget license
%define product_groupUtilities
%define install_prefix/usr

BuildRoot: %{_tmppath}/%{product_name}-build-root-xxxxx
Release: %{rpm_release}
Summary: the ultimate tool
Name: %{product_name}
Version: %{product_version}
License: %{product_license}
Group: %{product_group}
URL: %{product_url}
Vendor: %{product_vendor}

Source0: %{product_tarball}
Source1: widget.conf-DEFAULT

Patch0: widget-patch.001.local-tweaks
Patch1: widget-patch.002.backported-bugfixes

The Widget tool does exactly what it should, no more, no less.

As such, your system is incomplete without it.

* Fri Nov 14 2003 Gabriel
- added I18N/L10N fixes for French locale

* Mon Nov 10 2003 Moose
- added patch for site-specific customizations
- incorporated backported security fixes from newer version

rm -rf ${RPM_BUILD_DIR}/%{product_nameversion}
zcat ${RPM_SOURCE_DIR}/%{product_tarball} | tar xvf -
%patch0 -p0
%patch1 -p0
cd %{product_source_dir}
./configure –prefix=%{install_prefix}

cd ${RPM_BUILD_DIR}/%{product_source_dir}

rm -rf ${RPM_BUILD_ROOT}
make install ALT_ROOT_DIR=”${RPM_BUILD_ROOT}”
mkdir -p ${RPM_BUILD_ROOT}/%{doc_dir}
cp LICENSE ${RPM_BUILD_ROOT}/%{doc_dir}
cp README ${RPM_BUILD_ROOT}/%{doc_dir}
cp widget.SAMPLE_CONFIG ${RPM_BUILD_ROOT}/%{doc_dir}
cp widget.%{product_man_section}.html ${RPM_BUILD_ROOT}/%{doc_dir}
cp widget.%{product_man_section}.txt ${RPM_BUILD_ROOT}/%{doc_dir}
mkdir -p ${RPM_BUILD_ROOT}/etc
cp widget.conf-DEFAULT ${RPM_BUILD_ROOT}/etc/widget.conf



%dir %attr( – , root , root ) %{install_prefix}
%dir %attr( – , root , root ) %{install_prefix}/sbin
%dir %attr( – , root , root ) %{install_prefix}/etc

%attr( 0500 , root , root ) %{install_prefix}/sbin/widget
%conf %attr( 0400 , root , root ) %{install_prefix}/etc/widget.conf

%dir %attr( – , root , root ) %{install_prefix}/man
%dir %attr( – , root , root ) %{install_prefix}/man/man\

%attr( 0444 , root , root ) %{install_prefix}/man/man\

%doc %dir %attr( – , root , root ) %{install_prefix}/share
%doc %dir %attr( – , root , root ) %{install_prefix}/share/doc
%doc %dir %attr( 0555 , root , root ) %{install_prefix}/share/doc/\
%doc %attr( 0444 , root , root ) %{install_prefix}/share/doc/\
%doc %attr( 0444 , root , root ) %{install_prefix}/share/doc/\
%doc %attr( 0444 , root , root ) %{install_prefix}/share/doc/\
%doc %attr( 0444 , root , root ) %{install_prefix}/share/doc/\
%doc %attr( 0444 , root , root ) %{install_prefix}/share/doc/\

When you run rpmbuild, here’s what happens (roughly):

1. Your original, pristine source code bundle (stored as a tarball) and any supplemental files are unpacked into a temporary working directory.

2. If you’ve specified any patch files, those patches are applied to the source.

3. rpmbuild executes the configure, make, and make install in three separate phases. The make step runs in a temporary working directory, and it’s typical to point make install to an empty staging area.

4. Finally, rpmbuild collects the files and directories from the staging area into the RPM. The spec file details what files and directories to include, and also specifies what permissions, owner, and group should be set on each file or directory when the RPM is installed on the target system. Some files can even be tagged as configuration files, so that an upgrade to a later version of the sofwtare doesn’t overwrite those files.

Changing Your Build Process for RPMs

Building an RPM may require you to make changes to your Makefiles and installation routine.

A typical build process includes a step to install the software. In Makefiles, this is usually achieved through the make install target. Here’s an example:

cp widget /opt/tools/bin
cp widget.1 /opt/tools/man/man1

While (over)simplified, this install target boldly assumes that the files to be installed do not exist on the build system, or in the very least, that the files can be overwritten with impunity.

rpmbuild installs your files before it packages them into an RPM. To prevent clashes with existing files and to permit non-root users to build packages (think “damage control”), you must update your Makefiles to (optionally) install into a staging tree. For example, you might update your Makefiles’ install target to look like this:

cp widget ${ALT_ROOT}/opt/tools/bin
cp widget.1 ${ALT_ROOT}/opt/tools/man/man1

The product will be installed in the default location /opt/ tools/ if you run make install or to the staging area /tmp/ build/ if you run make install ALT_ROOT=”/tmp/ build”.

${ALT_ROOT} is used purely for packaging purposes, and is not defined in the Makefile. Instead, if you want to change the target directory of make install, provide its value on the command line, as shown above.

If you are repackaging someone else’s product, remember that RPM philosophy encourages the use of pristine source code. Instead of changing the Makefile directly, you should create a patch.

Spec files comprise three sections: a preamble, a list of scripts, and a file list.

The preamble lists product information, macros, dependencies, and auxiliary files. Product information includes the product’s name, version, and a brief description. Most of the fields are self-explanatory, but the Release: tag refers to the RPM itself and not the product packaged therein. If you were to release an updated RPM for the same product (say, to amend the file list), you would increment the Release: number.

Macros, created with the %define prefix, are used like variables and centralize changes to reduce errors. The sample spec file in Listing One makes generous use of macros. (Here’s a helpful tip: before you reach for autoconf to handle the macro definitions, realize that the spec file and your archived source code exist as separate entities at build time. Since your source code is still archived, autoconf cannot be run, and hence it cannot update the spec file.)

Dependencies enumerate prerequisites for building or installing your product, and also identify those products that are incompatible with yours. The Requires: tag lists dependencies for installing the product, whereas BuildRequires: lists prerequisites for rebuilding your product from the SRPM.

Auxiliary files include patches to be applied to the source tree (labeled PatchN, where N is a number) and files to be included in the RPM (labeled SourceN) that are not necessarily built or installed by the Makefile (such as an init.d script or sample configuration file.)

Scripts control rpmbuild: prepare the build tree, compile the software, install the product to the staging area (${ALT_ROOT} in the examples), and clean up all of these files once the RPM’s been successfully created. All scripts are defined right in the spec file, and are executed as Bourne shell scripts. (See rpmbuild‘s documentation for the complete list of predefined environment variables available to scripts.)

The scripts are called in the following order:

1. The %prep script unpacks the source and applies patches. If the product uses autoconf or something similar, this is where you would call ./configure.

2. The aptly named %build script performs the build, such as running make.

3. The %install script populates the staging area by calling make install for make-based builds. [In Listing One, note the definition of ${ALT_ROOT} on the commandline.] This is also where you would copy any required auxiliary files (that is, SourceN) to the staging area.

4. The %clean script clears out the build root after the RPM’s been successfully created. Blindly trashing the build root, can cause damage if this variable is mis-set. If you perform your RPM builds as a user other than root, though, this is a moot issue.

RPM supports other scripts that run alongside product installation and removal, but they are rarely needed. (Put another way: such scripts make numerous assumptions about the target environment, which is more than likely out of your control, so take heed when using them. Also, accomodating upgrade versus install versus concurrent install of the same product, if your RPM permits, is not a task to be taken lightly.)

Each of %prep, %build, and %install is turned into a separate script. That means environment variables (other than the predefined RPM variables) or cd commands issued in one script do not carry over to the next script. Additionally, each script statement is tested for success. Any failures — return codes other than 0 — cause rpmbuild to fail.

As shown in Listing One, rpmbuild extracts the source archive into a subdirectory of ${RPM_BUILD_DIR} (/tmp/rpmtest/BUILD in the example), which must be manually cleaned out if a build fails. You otherwise risk tripping up make and other tools that use a file’s existence as a dependency check on subsequent builds.

Finally, the %files section lists the files to be bundled into the RPM and, later, installed in the target environment. The names must exist under the ${ALT_BUILD_ROOT} directory after the %install script runs or rpmbuild will fail.

Tags specify permissions and other details in the file list:

* %perm( mode, owner, group ) sets the permissions for the item. A - used as an argument means that particular attribute should be left alone if the target already exists, or set to the respective attribute of the current user (the RPM builder, not the installer) if it doesn’t.

Directories not flagged with %dir are recursively included in the archive, causing harmless yet annoying File listed twice: warnings. Resist the temptation to specify / as %files‘s sole entry. Doing so prevents you from specifying fine-grained ownership and permissions.

In large-scale farms or barebones environments, supplementary documentation is unneeded. Flag readme files and the like with %doc so those files can be omitted from the install using rpm –excludedocs.

Creating a Source Tree Patch

Patches detail differences between text files, and are typically created by the Linux/Unix diff utility. When working with multiple source code files, it’s cleaner and simpler to create a single patch for the entire tree, rather than one patch per file.

To create such a patch, you need two, parallel source code trees. The first tree holds the original, pristine source straight from the tarball; the other tree is a copy of the first tree, except for any modifications you’ve made. For example, given the product widget-1.0, you might have the two trees:


Given the two trees, run diff to generate the patch:

$ diff -r -u widget-1.0 widget-1.0-CHANGES > \

It’s important that the directory in which you made your changes be listed second on the command line or else the subsequent patch operation will fail.

*Files marked with %config are preserved during upgrades or reinstalls, which is ideal when you expect such a file to be customized in the target environment. (You may recognize the .rpmnew and .rpmsave suffixes.)

Any files found in the staging area that are not included in the %files list cause rpmbuild to fail. So, if make install copies files you don’t want in the RPM, remove them at the end of the %install script or face the consequences.

By default RPM attempts to compress manpages and info docs with brp-compress, so you must update your %files section accordingly.

Testing the Process

As simple as it looks, there are plenty of ways for an RPM build to go awry. So, it’s best to test, test, and re-test.

Create a sandbox, say /tmp/rpmtest/, in which to test your spec file (again, think “damage control”). During testing, make sure that all commands are run as your user ID, not as root.

To use your sandbox, add the line…

%_topdir /tmp/rpmtest

… in ${HOME}/.rpmmacros. This tells RPM where to look for its build-related directories. (Be sure to remove this when you’re done testing your RPM build.)

Next, create the following directories: /tmp/rpmtest, /tmp/rpmtest/BUILD, /tmp/rpmtest/RPMS, /tmp/rpmtest/ SOURCES, /tmp/rpmtest/SPECS, and /tmp/rpmtest/SRPMS, and copy your application’s source tarball and any auxiliary files (specified as SourceN or PatchN files in the spec file) to /tmp/rpmtest/SOURCES. Next, run…

$ rpmbuild -ba your_spec.file

… to create an RPM and SRPM of your product. rpmbuild is not shy about problems. Watch the screen trace and correct any mistakes in the spec file.

It can be time-consuming to repeat an entire build when debugging. Make use of rpmbuild‘s flags, such as -p, -c, and -i (see the man page for details), to take control of the process. Build up to the point that it breaks, then rerun only the part you’ve tweaked.

The Wrap-Up

RPM is a powerful package manager. This month’s column only touched upon the basics, but that should be enough to get you started. For more advanced RPM usage (such as subpackages and md5 checksums) refer to the RPM documentation.

Ethan McCallum is a freelance technology consultant who weaves a web of C++, Java, and Linux to keep the housework at bay. He can be reached at ethanqm@penguinmail.com. A sample, annotated spec file may be downloaded from the Linux Magazine website at http://www.linux-mag.com/downloads/2004-02/compile.

Comments are closed.