After much anticipation, the Linux 2.4 kernel is finally ready to make its debut -- and with this release, Linux truly becomes "Enterprise Ready." We will fill you in on all of the new and improved features.
Some of you may remember that in January 2000 we wrote about the imminent release of the Linux 2.4 kernel. Well, unless you’ve been living in a cave for the past 12 months, you are probably aware that this expectation was somewhat premature. Almost a year later, the 2.4 kernel still has not reached “official” release status.
More than a few people have complained about the “lateness” of the 2.4 release. However, as any good kernel hacker will tell you, you can release software that’s good, software that’s inexpensive, or software that’s available on time. You can usually release software that has two of these three attributes — but not all three.
However, open source software seems to adhere to a somewhat different set of principles. Open source developers have a “release no software before its time” policy. Rather than shipping buggy code that doesn’t meet their standards, the Linux kernel developers would rather wait until “it’s done right.”
In any case, it would be unfair to accuse the kernel development team of a late release. They never promised us a kernel by Christmas — certainly not by Christmas of 1999.
However, as we go to press with this issue, it does appear that the long-awaited release of the 2.4 kernel is imminent. For several weeks now, Linus Torvalds himself has been indicating that the new kernel should be posted, “in early December,” or at worst, just before the end of this year.
By now, you’re probably wondering what all the fuss is about? Just what’s in this new kernel that’s causing all these delays? We’ll attempt to answer these questions by recapping the major features of the new kernel, as they were known in mid-November (at which point Linus announced a feature freeze, meaning that bugs would be fixed, but no more new features would be added to the final release).
Despite the delays, one thing is clear — Linux is moving upstream. As Linux is used in more and more enterprise environments, the kernel is changing to better fit into those environments. The 2.4 kernel contains much new or improved functionality that is aimed at enterprise IT users; these features fall into four main categories:
- Architectural changes
- Support for filesystems
- Support for networking
- Support for devices
Of course, there are many new features that are of interest to desktop and workstation users as well; we documented many of those in our previous article, available online at http://www.linux-mag.com/2000-01/linux2-4_01.html.
Since we have already looked at the new kernel from the desktop/workstation point of view, we will now focus on the four main areas of the kernel that have changed, and how those changes will affect those who wish to use Linux as an enterprise or server OS.
Some of the most important changes in Linux 2.4 relate to hardware and software architecture. With 2.4, the kernel now supports as much as 64 GB of memory on Intel x86-compatible systems that support physical address extension (PAE), which includes systems based on the Pentium Pro and subsequent processors. While few desktop users will find themselves needing to use 64 GB of RAM, enterprise-class systems that must support thousands of simultaneous users demand more memory than the comparatively paltry 2 GB supported by Linux 2.2.
The ability to use lots of RAM should also be a major boon to those who wish to use Linux with large databases. Database query and access times drop significantly when more of the database can be located in RAM rather than being fetched from disk.
Linux 2.4 also increases and improves Linux support for symmetric multiprocessing (SMP). Linus reports that it is “painful” for him to go back to using the 2.2 kernel on multiprocessor machines. 2.4 should be optimized for machines with up to eight processors, but some parties (notably Linuxcare) are reported to have 2.4 working on machines with up to 64 processors.
In order to enable this sort of multiprocessor scalability, task synchronization and threading facilities have been enhanced to increase the efficiency of Linux-based SMP. Consequently, x86-based Linux boxes are likely to begin challenging traditional “big iron” boxes in enterprise applications (the sort that typically demand enormous computing power). It’s easy to see why big iron companies, like IBM, have developed a keen interest in Linux, which may soon invade their traditional domain, the mainframe data center.
Linux will doubtless be one of the first operating systems to support Itanium, Intel’s powerful 64-bit processor (formerly Merced). Although the Itanium is not being widely distributed quite yet (leading some to dub it “Unobtanium”), Linux 2.4 has run successfully on demonstration Itanium chips provided by Intel. So, while some tweaks may still be required, Itanium support is essentially complete. An SMP array of 64-bit Itanium chips running Linux would indeed be a formidable computing platform, capable of more than 24 billion instructions per second.
Linux 2.2 provided supported for some special memory transfer instructions included in Intel’s Pentium Pro and subsequent processors. Linux 2.4 adds similar support for processors manufactured by AMD, Cyrix, and WinChip.
These instructions expedite movement of data from one region of memory to another and significantly increase effective processor speed. Desktop and enterprise users alike will benefit from this support, provided they’re running software optimized for these processors.
The new kernel also supports multiple input/output advanced programmable interrupt controllers (APICs). An SMP system board that is equipped with multiple APICs is able to spread interrupt processing among its CPUs, resulting in faster, more granular multiprocessing.
The architecture of the Linux kernel itself has been greatly improved in several important respects. For example, Linux 2.4 now provides 32-bit user and group IDs and can therefore handle as many as — wait for it — 4.2 billion users and groups!
Admittedly, even enterprise users are unlikely to configure 4.2 billion users any time soon (unless of course they are issuing user IDs to insects and rodents). But the 65,536 user and group IDs provided for by the 2.2 kernel with its 16-bit value had proven insufficient for high-end users. Now everyone can happily assign as many user and group IDs as they need to without ever having to worry about running short.
The last significant architectural improvement in Linux 2.4 is that the limit on the maximum number of processes that can be running concurrently is now specified at run time rather than when compiling the kernel. Rather than being limited to an arbitrary value such as 1024, the number of concurrently running processes is now effectively limited only by the amount of available memory. Since enterprise applications are notorious for requiring enormous numbers of processes, this is a welcome improvement.
System Architectural Features
- Support for additional CPU architectures
- Support for large memory
- Support for special instructions
- Support for multiple APICs
- Support for large numbers of users and groups
- Support for run-time process limits
Linux’s filesystems contain a host of new improvements, ranging from increased efficiency to added functionality. Among the most significant improvements, the virtual filesystem (VFS) has been rewritten to use only a single cache (rather than separate read and write caches).
Working with two caches had complicated kernel development and was the source of several troubling bugs. By combining the two caches, kernel developers have made the VFS code much simpler to understand and, therefore, more robust. Moreover, elimination of the redundant cache reduces memory requirements. Both desktop and enterprise users can expect greatly improved file input/output performance as a result of the improved VFS efficiency.
Beyond becoming more efficient, the filesystem seems to be gearing up and getting ready for Linux’s role as a database server. One limitation of the 2.2 kernel that was troubling to enterprise users was the 2 GB file size limit on 32-bit systems. Linux 2.4 eliminates this restriction, allowing the creation of files as large as 16 terabytes. It will be a while before most users — even enterprise users — run up against this rather large new file size limit.
The new kernel also supports raw input/output devices that entirely bypass the caching provided by the VFS and kernel. Raw input/output devices are useful for special applications, such as database management systems, that provide their own caching. For such applications, raw devices eliminate the overhead otherwise associated with double caching. Enterprise users of database management systems, and users of other specialized input/output intensive applications, can expect improved performance when such applications are revised to use the new support for raw devices.
Many users had hoped that Linux 2.4 would include advanced filesystem capabilities such as journaling. Journaling is especially important to enterprise users who maintain large datasets. Checking the integrity of an ordinary Linux filesystem is time consuming, particularly for larger filesystems. Making matters worse, traditional Linux filesystems (such as ext2) are not especially fault tolerant.
While development of journaling filesystems is ongoing, the standard Linux 2.4 kernel will not actually include one. Nevertheless, some Linux distributions will likely ship with a kernel supporting a journaling filesystem. For example, some recent releases of the SuSE distribution have included a kernel that is patched to support reiserfs, which provides both journaling and enhanced in-put/output performance. Other advanced filesystems include jfs,ext3, and xfs.
Ext3 is a journaling filesystem based on the familiar ext2 filesystem and has progressed significantly in the past several months. For the pre-release source code of ext3, see ftp://ftp.uk.linux.org/pub/linux/sct/fs/jfs/ (please note that although this URL seems to contain the jfs source code, it is in fact the correct URL for ext3, not jfs.)
Incidentally, Stephen Tweedie, the primary author of ext3, is working for Red Hat. However, Red Hat has not announced definite plans for including a journaling filesystem in future releases of Red Hat Linux.
The August issue of Linux Magazine contained an article on Journaling Filesystems that discussed all of these issues in greater detail; it is now available online at http://www.linux-mag.com/2000-08/journaling_01.html.
Finally, Linux 2.4 also provides support for Logical Volume Management (LVM). LVM allows users to combine and partition physical disks and partitions as one or more logical disks and partitions. This functionality is especially useful to enterprise sysadmins who must configure new devices without the benefit of taking down active systems or datasets.
The Guru Guidance column in our November 2000 issue explained in detail LVM and the tools associated with it. If you don’t have the issue, don’t worry about scrounging up a back issue; it’ll soon be available at http://www.linux-mag.com.
- Improved VFS layer
- Larger file size limit
- Raw input-output devices
- Journaling filesystem
- Logical volume management
Another Linux 2.4 subsystem that has been modified to improve performance is the networking layer, which has been rewritten and shows tremendous performance improvements while multitasking. The Linux 2.2 kernel supported multiple network interfaces rather poorly. Performance dropped off dramatically on multi-interface systems, a fact that was amply demonstrated by the notorious benchmarks conducted by Mindcraft in the spring of 1999.
Partly because of those benchmarks, Linux kernel developers made several notable improvements that increased networking efficiency, especially for using multiple network interfaces and using multiprocessor (SMP) systems. For example, under Linux 2.2, when a network input/ output event occurs, the kernel must wakeup all the processes that are sleeping in order to determine which process is actually waiting for that event.
The result is that many processor cycles are wasted. In contrast, Linux 2.4 awakens only the process that is waiting for the event that occurred. Thus, the 2.4 kernel’s networking performance scales much better than that of 2.2.
Linux 2.4 also provides a kernel module called the kernel HTTP daemon, or khttpd, which serves static Web pages and can work with the Apache Web server or another Web server to serve dynamic Web pages. No extensive benchmarks have yet been published, but khttpd will likely help Linux win many Web server benchmarks.
Linux’s implementation of the network file system (NFS) has been only partially compatible with version 3 of NFS (as distributed by Sun Microsystems). The Linux 2.4 kernel’s support for NFS is much more compatible with NFSv3. This will ease the burden on Linux sysadmins who happen to maintain heterogeneous environments which include Linux and other operating systems that are running NFSv3. Linux 2.4 also includes support for DECnet and ARCnet, similarly easing the burden on sysadmins responsible for networks based on these technologies.
Linux has for some time been the only operating system compliant with the full IPv4 specification. Changes to the Linux 2.4 TCP/IP implementation make it more efficient and scalable than the Linux 2.2 release. One price of IPv4 compatibility is the removal of support for the colon-mode syntax used to specify IP aliases. This support was deprecated but still present in Linux 2.2. Those who have scripts using the obsolete syntax must revise them to configure aliases via the ip command before the script will operate under Linux 2.4.
The Linux 2.2 kernel replaced the ipfwadm facility used to construct firewalls with a new ipchains facility. In turn, Linux 2.4 replaces ipchains with iptables, which is also known as netfilter. More information about netfilter can be found online in the January 2000 (http://www.linux-mag.com/2000-01/bestdefense_01.html) and June 2000 (http://www.linux-mag.com/2000-06/gear_01.html) issues of Linux Magazine.
In comparison to its predecessors, netfilter provides a few additional capabilities that permit construction of more sophisticated firewalls. Although you can access netfilter’s capabilities via the command line, it’s also possible to write and use user-space modules to configure packet filtering rules via netfilter. Eventually, this will likely lead to more convenient ways to configure sophisticated fire-walls. netfilter can also be used to implement network address translation (NAT) for support of masqueraded private networks. If you prefer to continue using the more familiar ipfwadm or ipchains syntax, you’ll be grateful to learn that netfilter includes a compatibility feature that can accommodate the syntax of these older facilities.
- Improved networking layer
- Kernel HTTP daemon
- Improved NFS compatibility
- Support for IPv4, IPv6
- Logical volume management
- Support for DECnet and ARCnet
- Improved NAT performance
- Improved firewall support
Of course, every new release of the Linux kernel includes new and improved support for more devices, and 2.4 is no exception. First, 2.4 allows you to use more devices than was possible under 2.2. This will allow enterprise users to properly configure the bigger boxes needed for industrial-strength computing. For example, you can now use up to 10 IDE controllers concurrently and as many Ethernet cards as you want. Under Linux 2.2, you were limited to four IDE controllers, and Ethernet card usage was constrained by the limited support for multithreading in the networking layer.
If you’re a desktop user, you may wonder why anyone would want to use more than a few Ethernet cards. But, for example, a system using 32-way SMP might need a network interface for each CPU. Likewise, a system configured as a multi-port router might need many network interfaces (one for each port).
The 2.4 kernel’s networking implementation exacts an overhead of only 2 KB per network interface. In principle, you could configure a system with dozens of network interfaces but you’d likely run out of slots, and your CPU would likely be overtaxed well before then. So under Linux 2.4, only your system’s resources effectively limit the number of network interfaces.
If Ethernet is not your thing, or you need more than Ethernet alone, Linux 2.4 provides improved support for ATM (Asynchronous Transfer Mode) network adapters, which are important in high-bandwidth apps such as maintaining Internet POPs. An ATM adapter can handle bandwidths as high as those provided by OC-12 connections, which reach 622 Mbps.
Beyond networking devices, Linux 2.4 includes optimized support for SCSI-2 devices, such as the Tagged Command Queuing (TCQ) feature that allows the host SCSI adapter to send multiple commands to a device queue without waiting for completion of previous commands. This can greatly improve the performance of compatible SCSI devices in both desktop and enterprise applications. The new kernel also supports several RAID devices (for which support was backported for some of the later releases of 2.2). Due to their large capacity and high reliability, RAID devices are the bread and butter of enterprise data storage.
The 2.4 kernel also supports the increasingly important Firewire interface, also known as IEEE 1394. Desktop users will find Firewire convenient for accessing video data at rates not possible via the more popular USB interface. Firewire can transfer data at ranges from 100- 400 Mbps, depending on the capabilities of the interface. In comparison, current USB implementations transfer data at no more than 12 Mbps. Enterprise users will find Firewire appropriate for process control applications. Although Linux is not a true real-time operating system, its performance is adequate for many less demanding real-time applications.
- Support for 16 ethernet cards
- Support for 10 IDE controllers
- Support for new SCSI controllers
- Support for Firewire (IEEE 1394)
- Support for ATM controllers
This brief article has afforded only a taste of what’s available in Linux 2.4. Those who are interested in learning more about the Linux 2.4 kernel, or about the suitability of Linux for enterprise applications, have a resource. A white paper has been prepared by the staffs of Veritas and Linuxcare, which is called Giving Linux Enterprise Strength. The paper is available online at http://www.linuxcare.com/about-us/collateral/whitepapers/linux24-enterprise.pdf.
Although the 2.4 kernel was subject to a feature freeze at the time of this writing, it’s possible that the final release will differ slightly from the description given here. Be sure to check the release notes to learn exactly what features are included. To obtain a copy of the kernel, visit http://www.kernel.org.
Bill McCarty is Associate Professor of Web and Information Technology at Azusa Pacific University. Patrick McCarty is an enterprise-level Linux sysadmin and RHCE.