Last month, we talked about writing portable code and focused on how to use feature test macros, coding standards, and emulation of uncommon functions to make porting even easier. This month, we'll talk about portable build systems -- tools you can use to easily build that portable code on a variety of target platforms.
Last month, we talked about writing portable code and focused on how to use feature test macros, coding standards, and emulation of uncommon functions to make porting even easier. This month, we’ll talk about portable build systems — tools you can use to easily build that portable code on a variety of target platforms.
What’s a Build System?
A build system is the collection of components needed to turn your source into a distribution. Source can just be code, and a distribution can be nothing more than single executable. In that case, a compiler may be the build system.
For example, it’s entirely possible that your application is simple enough that the following command could build it all:
However, most interesting software is more complex than that. Source typically includes code (perhaps in multiple programming languages), man pages, shell scripts, configuration files, and data. Given such complex source, a common build system typically contains compilers, linkers, libraries, include files, and a suite of supporting utilities. Distributions usually include installers, documentation, multiple executables, and sometimes even another build system (for example, if you’re distributing source, the source code and its build system is your distribution).
Undoubtedly, the mention of “build system” brings the ubiquitous make to mind. make is certainly a very common and important component of build systems, so we’ll spend most of this month’s column discussing it, the many make variants, and how to choose one variant over the other. However, keep in mind that make is really a small part of the entire build system — a driver if you will. Building a portable build system requires many decisions, not just the simple one to use make.
Certainly, the design of a build system depends on the project at hand. As we saw above, a build system can just include gcc, if that’s sufficient. The design of a build system also depends on the intended audience. It’s one thing to create a build system for developers and quite another to create one for customers.
For developers, the build system is integral to their jobs. The developers want to avoid any unnecessary software rebuilds, but also want to be sure that modules are rebuilt when necessary. For example, in a C project, if an important .h file changes, all of the .c files that use that include file should be recompiled.
Customers, on the other hand, are unlikely to care that they can avoid rebuilding everything each time something changes, because nothing is going to change for them. It’s likely that they’re going to unpack the distribution once, configure it once, build it once, and install it once. For customers, a long, simple, shell script might suffice as the build tool.
What’s in a make?
The most common “build system” used on Unix-like systems is certainly make. make creates dependency trees and selectively rebuilds portions of the source depending on what changes were made.
Currently, there are three dominant versions or flavors of makes in wide use: GNU make, from the Free Software Foundation, and widely available on Linux; Berkeley make, derived from 4.4 BSD, and available on FreeBSD, NetBSD, and OpenBSD; and System V/Single Unix Specification (SUS) make, available on Solaris, HP-UX, and other System V derived systems. (There are also several minor flavors of makes available, but none are as widely used.)
GNU make and Berkeley make are both proper supersets of SUS make. Thus, a Makefile written for SUS make will be interpreted correctly on both GNU make and Berkeley make, as well as on the System V variants. Sadly, the same cannot be said for either Berkeley make or GNU make. Both have a large number of extensions that are incompatible with SUS make and with each other.
For instance, while all versions of make include a mechanism for including other files into the currently running Makefile, each version implements it differently and incompatibly. Unfortunately, the incompatibilities render an extremely useful mechanism for sharing configuration information across multiple Makefiles useless. However, it is something that can be worked around, using make‘s own facilities. To find out how, see “Making make Include a File.”
Making make include a file
The Single UNIX Specification edition of make doesn’t document an include feature, leaving that as a vendor extension. SUS doesn’t include the facility because historic implementations of make have evolved using different include mechanisms, and there are still a few versions of make out there that don’t support includes at all.
There are ways around this limitation. GNU’s Automake supports a mechanism for including files into it’s Makefile.in output. We’ll discuss Automake and Autoconf next month.
Another way to workaround the limitation is to use the text preprocessors available on Unix systems. The most common preprocessor is cpp, the C pre-processor. The X Window System uses cpp as the heart of imake to generate Makefiles for projects using X.
Another preprocessor, m4, is actually very powerful because it supports text substitution and macro expansion. The most common use of m4 these days is for building sendmail configuration files, but Automake and Autoconf also use m4 at their core.
Here’s how to use m4 directly to generate Makefiles. Assume the following set of files and directories:
Listing One shows the initial progA/Makefile.m4 template and Listing Two shows the contents of the top-level Makefile.
The m4 directives in Listing One are include(), tchangecom(), and dnl.
include() does pretty much what you’d expect, suspending input from the current file, and reading from the named file until end-of-file is reached. Thus, our final Makefile contains the comments of the files ../common/rules.m4, itself, and ../common/sources.m4.
changecom() changes the comment delimiters used by m4. By default, m4 uses the same comment delimiters as nearly every other interpreter on Unix, the octothorpe or “pound sign” (#) and new-line. Since we’d like to leave the comments in the resulting Makefile, using changecom() allows us to change m4‘s comment character to something else. In this case, we turn it off entirely.
The last m4 directive we use is dnl, which stands for “delete to new-line”. It suppresses blank lines where the m4 directives are processed. It’s not strictly necessary, but leaves the output looking a little prettier.
progB/Makefile.m4 looks similar to progA/Makefile.m4.
Using m4, you’ve successfully emulated the include feature of the different make flavors, without depending upon a particular one.
To ensure maximum portability of Makefiles, the definition from SUS should be used. SUS has worked extremely hard to create a proper subset of all the currently existing make formats.
If your project must use the features of one of the more extended makes, be sure to call out the dependency in the project’s documentation (README or INSTALL are common places to flag this kind of thing). On the other hand, if the project is using only features as defined by SUS, adding the target .POSIX would be a good idea. Declaring .POSIX tells various vendor implementations that your project is using only features defined by SUS, and to turn off features that might conflict.
Listing Two: A Makefile built with m4
#@(#) ./Makefile — master makefile for ProjectZ
# default target and dependencies
all:progA/Makefile progB/Makefile build
build: progA/a.out progB/a.out
cd progA; $(MAKE)
cd progB; $(MAKE)
progA/Makefile: progA/Makefile.m4 $(COMMON_MK)
cd progA; m4 Makefile.m4 >Makefile
progB/Makefile: progB/Makefile.m4 $(COMMON_MK)
cd progB; m4 Makefile.m4 >Makefile
Webster’s Dictionary (http://www.webster.com) defines dependent as:
1. hanging down
2.a. determined or conditioned by another : CONTINGENT
2.b.1. relying on another for support
2.b.2. affected with a drug dependence
2.c. subject to another’s jurisdiction
3. SUBORDINATE: not mathematically or statistically independent
For the purposes of software development, definitions 2a, 2b.1., 2c, and 3a all describe a dependency. Fundamentally, file A is dependent upon file B if changing B requires A to be rebuilt.
The key to using make well, no matter which make you use, is defining accurate dependencies. Dependencies seem like such an obvious thing, but they can be very painful to generate, and when improperly defined, can cause great headaches. Indeed, getting the source-object-executable dependencies correct can be the hardest part of writing a Makefile. For large projects, it can be extremely hard to do manually, at least if they are added late in the game.
Fortunately, tools exist to help generate file dependencies. mkdep, gcc and other tools exist for preprocessing sources and emitting a list of files included by each source module.
To use gcc to generate a list of dependencies for the source file foo.c, use the following command:
The command will emit the dependency list to standard output. The output should be redirected to a file, perhaps the Makefile itself. Here’s an example of doing that:
gcc -MM foo.c >> Makefile
Other compilers support the -M flag, which lists all files included, both system header files and user files.
mkdep performs a similar task, generating the file .depend to store dependency information. You can then include .depend in the Makefile.
One note: you should cull the dependencies generated by gcc and mkdep to remove system include files since they’re unlikely to change during the course of development. Once culled, include the list of dependencies in the Makefile.
A good Makefile should also include three standard targets. The first is all, and it should be the first target in the Makefile. all should be dependent on the executables (and anything else to be installed). Having all as the first target in the Makefile means it’s the task chosen when no specific target is defined on the command line.
The second standard target is install. The install target should be dependent on anything that needs to be installed for your distribution to work. install should copy the dependents into the appropriate directories on the system, optionally allowing someone to relocate the installation root using the make variable DESTDIR.
The final standard target is clean. The clean target should remove all the detritus that’s left behind as part of building the software. This would be any intermediate files, such as object files (.o‘s), and the executable(s). The clean target should leave the directory similar to its initial checkout/un-archived state. Your local coding standard may mandate other standard targets as well.
To recap, follow these guidelines to make your build system as portable as your code:
- Avoid using vendor specific make extensions as much as possible. Use the SUS make definition to write your Makefiles.
- Make your Makefiles as readable as your code. Document your system dependencies (compilers, libraries, etc.)
- Maintain the file dependencies in your Makefile. Use tools like gcc -MM or mkdep to generate them if necessary.
In general, by restricting your Makefile to the SUS feature set, your project can be built on any system from V7 on.
Eric Schnoebelen is a software developer with 20 years experience in both operating system and applications development. Eric can be reached at firstname.lastname@example.org. The templates shown in this column can be downloaded from the Linux Magazine download page at http://www.linux-mag.com/downloads/2002-10/compile.