Why Linux on Clusters?
Linux on high-performance computing clusters seems an obvious choice now, but it wasn’t a forgone conclusion when Thomas Sterling and Donald Becker used Linux to build the world’s first Beowulf cluster in 1999. Linux has come a long way since then. Learn why Linux has put “super” back into supercomputers.
Monday, November 15th, 2004
Over the past few years, high-performance computing clusters — tightly- or loosely-coupled collections of low-cost, off-the-shelf computers — have largely supplanted proprietary supercomputers. Even with hundreds or even thousands of compute nodes, a commodity hardware HPC cluster is a fraction of the expense of the likes of IBM’s Blue Gene or Fujitsu’s RIKEN Super Combined Cluster.
At the same time, Linux features have been greatly expanded and refined. Today, Linux is as capable as any Unix, and because Linux source code is widely and freely available, Linux has also been customized to scale up or scale down to virtually any computer configuration.
Perhaps obviously, the combination of HPC clusters and Linux has become the de facto standard for solving complex computing problems. In academic and research environments, the open, unencumbered, even “home grown” spirit of the two solutions holds great appeal to bleeding-edge yet often budget-constrained scientists.
But what is it about Linux that has made it so successful in HPC? And more importantly, what can its success tell us about how we might approach HPC in the future? Let’s find out.
In the Beginning …
One of the first HPC efforts to use Linux was The Beowulf Project. (For more information, see the sidebar “What is a Beowulf?”) According to project founder Thomas Sterling, Linux wasn’t the most capable operating system in 1993, the year Beowulf was launched, but it was the most appealing:
