Everyone professes to write portable code, but few programmers actually manage to do it. In most cases, so-called portable code comes out littered with #ifs or #ifdefs (or worse, nested #ifs and #ifdefs), rendering the code illegible and obfuscated. Sadly, the lion's share of many porting efforts is spent trying to figure out which lines of code are actually being compiled and executed. This wastes time and energy, and can be downright frustrating.
Everyone professes to write portable code, but few programmers actually manage to do it. In most cases, so-called portable code comes out littered with #ifs or #ifdefs (or worse, nested #ifs and #ifdefs), rendering the code illegible and obfuscated. Sadly, the lion’s share of many porting efforts is spent trying to figure out which lines of code are actually being compiled and executed. This wastes time and energy, and can be downright frustrating.
To avoid porting hassles, many developers have switched to scripting languages like Perl or Python that tend to hide system specifics very well. However, if you’re writing the next Perl or Python interpreter, chances are you’re using a language like C or C++ and don’t have the luxury of scripting’s layers of abstraction.
To be sure, writing portable code is hard. Even if you’re just targeting Unix variants, there are a multitude of differences — in hardware, system calls, and programming interfaces — that you have to contend with.
Over the next few months, we’re going to delve into techniques and tools that can help you write and distribute extremely portable code. We’ll look at portable build tools, automatically generated test suites, cross-platform programming, and packaging systems like RPM, used to distribute source and binaries in a uniform way. Ultimately, we’ll see how to create a code release worthy of being called “open source.”
This month, we start the series with writing portable code.
What’s in an OS?
If you’ve been developing Unix software for even a little while, you’ve probably seen code something like this:
#if defined(__SOLARIS__) || (defined(__LINUX__) && __LINUX__ > 0204013))
#elif defined(__NetBSD__) || defined(__FreeBSD)
Snippets like this are compile-time multiplexers: they choose the appropriate code and flavor of application programming interface (API) based on the type and version of the local operating system. This (somewhat contrived) snippet reflects that Solaris and a specific version of Linux use one kind of API, while FreeBSD and NetBSD use another; a third conditional section of code (associated with #else) is used for all other cases.
While it seems extremely appealing and natural to test operating system type to determine if a specific API is available, that technique is probably the least portable and the most confusing to read. In particular, testing operating system type tells you little about the purpose of the code or its system dependencies.
For example, if you’re trying to port the code snippet above to a new platform, can you easily tell which of the three conditional code fragments, if any, is the most appropriate for your machine? Probably not.
A better technique is to test for the API (what we’ll also call a feature) that your code depends on itself. For example:
# error “__FILE__ needs gethostname_r or gethostname”
Here, you can easily tell that the code depends on the gethostname() feature. Moreover, the code prefers the re-entrant call gethostname_r() if it’s available. If gethostname() is not supported at all, the compiler emits an error. (System-specific #defines like #define HAVE_GETHOSTNAME 1 can usually be found in a .h file generated by something like MetaConfig or autoconfig. In an upcoming column, we’ll see how to use these utilities in your projects.) Of course, if you’re industrious, you can also replace the #error with an implementation of your own (see the sidebar “Fake It To Make It”).
If you’re developing some code and find a feature of your local operating system that you’ve just got to have, but discover it isn’t specified in the standards documents (or is unlikely to exist on other platforms), write your own implementation. An excellent example of this can be found in C News, written (over 10 years ago) by Henry Spencer and Geoff Collyer.
symlink() is very useful on systems running USENET News software because it saves critical disk space. For example, the original C News code used symlink() to save space whenever an article was simultaneously posted to multiple newsgroups stored on different file systems. However, in 1989, only BSD-derived systems regularly implemented symlink(). Strict System V (release 2 and 3) systems didn’t implement it at all. By applying some careful thought, Spencer and Collyer were able to optimize their code and make it portable.
First, they implemented a very simple symlink() for all of the systems that do not support that system call. The code looks something like this:
/* symlink dummy */
char *n1, *n2;
Next, they modified their algorithm to work in stages, from “best” case to “worst.” First, their algorithm attempts to make a hard link (using link()) to the original posting. If link() fails (probably because the link is from one file system to another), the code calls symlink() to make a symbolic link to the file (more expensive because it consumes another inode and at least one file system block, but works across file systems). Finally, if symlink() fails, the routine attempts an outright copy of the file (most expensive because it consumes an inode and additional file system blocks, but works in all cases.)
As you can see, the dummy symlink() always fails. So, on systems without a native symlink() system call, C News always reverts to the “worst” case and makes physical copies of cross-postings. Spencer and Collyer’s solution is rather elegant: it’s modular, encapsulated, and simple. And rather than reinvent the wheel entirely, the two programmers leveraged existing system calls on capable operating systems whenever possible.
If there are standards-based interfaces that provide partial or complete implementations of a feature, use them (with appropriate #if feature tests). If you write your own implementation of a feature, comply with the standard interface. And, if you cannot comply or if no standard exists, write a new interface that makes coding simpler and hides the underlying implementation.
If you’d like to see an example of many of the techniques described here, download a copy of C News from ftp://ftp.cs. toronto.edu/pub/c-news/c-news.tar.Z.
This technique is superior because it specifically calls out the system dependency, rather than requiring the reader to infer it. At a glance you can see the feature required and the intent of the code. To ease porting, you can even generate a list of dependencies with a simple command such as:
% grep ‘^#if[[:space:]]defined(.*)$’ *.[ch] | \
sed -e ‘s/^\(.*\):.*defined(\(.*\))$/\1: \2/’\ | sort | uniqX
This command creates a list of #defines used in #if preprocessor statements, along with source file names (note that this grep expression does not process complex #if conditions with && and || such as #if defined(HAVE_TERM_H) && defined(HAVE_CURSES_H), but you can easily extend it). Running the command on MySQL source generates:
Given a list like this, writing a porting guide should be easy or obvious: just document each dependency and what it does.
Hoist the Standard
Speaking of documentation, you’d think that the man pages installed on your local development system would be your best source of information. That’s not necessarily the case. Indeed, if you’re trying to write portable software, using your local man pages is an exceptionally bad choice. The man pages might describe your system’s APIs, but the goal is to write code that works on all APIs equally well.
A much better source of information on widely available interfaces is the standards documents, such as POSIX.1 (http://www.pasc.org). A standards sub-body of the IEEE, POSIX’s role is to standardize and document the API available on Unix(-like) systems. Admittedly, POSIX provides a restricted set of functions, but those functions are guaranteed to be implemented on any system claiming POSIX conformance.
Following the POSIX standards are the ISO standards (which are based upon the POSIX standards), the System V Interface Definition (SVID, http://www.caldera.com/developers/devspecs), or its successor, the Single UNIX Specification (SUS, http://www.unix-systems.org/version3/online.html), published by the Open Group, successor to the Open Software Foundation.
Another source of API specifications are the various web sites that provide online editions of system manual pages. For example, the FreeBSD (http://www.freebsd.org) and NetBSD (http://www.netbsd.org) sites provide quite a variety.
Bits, Nibbles, and Bytes? Oh My!
In this day and age, very few applications should care about natural bit and byte orders of a machine. Technology like XML provides a platform-independent storage medium, and you can generally use the features of a higher-level language to avoid minutiae such as the number of bits in an int or a long.
However, it’s important to remember that the world is not just Intel x86: off_t is not necessarily a C long, and C long is not always 32 bits. (For example, the Alpha architecture uses 64 bit longs and pointers. At one time, there was a Data General architecture that had 32 bit words, but 48 bit pointers. Even x86 processors have been capable of addressing 36 bits of address for a couple of generations now.) When using APIs, use the typedefs provided for you by the operating system. If you must have a data type of a known bit size, use typedef, and use the typedef consistently throughout your code (using the C99 typedef naming convention might even ease porting in the future.)
Here are some other guidelines to keep in mind:
- Avoid #if in mainline code. #ifs can obscure what the program is trying to do. Localize #if/#ifdef clauses to supporting modules.
- Encapsulate your data. This guideline may be obvious, but it’s probably worth restating. Define structures to contain related data elements, and then define manipulation functions for those structures. Hide as much as you can to allow the implementation specifics to change as required during the lifetime of the software.
- Organize your code. Keep each family of functions to a single source file. For example, keep all getter and setter functions for a custom typedef in the same file. And, make sure to keep your typedefs in a common source (include) file, which is included everywhere.
- Obey network order. If you’re communicating to a remote process, use nto*() and *ton() to ensure sure all network communications use network byte order.
- Avoid global variables. Enough said.
- Don’t splat. When writing data files, don’t just write a binary structure to disk. Instead, use a portable format, such as XML or even just comma separated ASCII strings. Chances are that the data structures will change. Again, abstracting the data today provides flexibility tomorrow.
Finally, in the spirit of portability and to make your code more readable, consider using a standard coding style style — for example, doc in the Linux kernel source tree seems to quite popular. What’s good for Linux (and Linus) is probably good for you.
Eric Schnoebelen is a software developer with 20 years experience in both operating system and applications development. Eric can be reached at email@example.com.