Beyond MPI
To date, the Message Passing Interface has been instrumental in simplifying application development for clusters. But as clusters change to embrace multiple cores, multiple platforms, and multiple advanced interconnects, MPI is no longer adequate. What can replace it? Donald Becker asks, “How about Unix? ”
Tuesday, November 15th, 2005
The well-known Message Passing Interface (MPI) is widely used in parallel programming and was intrumental in simplifying application development for clusters. However, as clustering evolves to leverage virtualization, MPI is becoming irrelevant: MPI doesn’t prevent version skews, cannot handle failovers, and lacks an efficient way for applications to communicate with each other — a critical capability for virtualization to be successful. Fortunately, there are emerging alternatives to MPI that are better suited to meet industry demands for cluster virtualization without adding unnecessary complexity.
A Narrow World View
For all of its power, MPI purveys a static view of cooperating machines. MPI assumes that software is installed once, at the same time, on all machines, and furthermore assumes that all machines are known and named when a cluster is installed or an application is started. By design, the MPI “cluster size” (provided by MPI_Comm_size(MPI_COMM_WORLD)) remains fixed after initialization, making it impossible to either take advantage of new machines added to the cluster or to reduce the number of machines on which an application depends.
The MPI model holds that the world doesn’t evolve — that applications are never modified and improved. In the real world however, application programs, configuration files, and system components are updated (often), and the most common thing to do after an update is to immediately utilize the changes. But because MPI is based on an initialize-compute-terminate model, explicit support for checkpointing, executing as a service, and running “forever” is lost.
