Parallel computing can be utilized to perform numerous computations quickly and solve problems by making many processors work simultaneously on smaller subtasks or subsets of the data. Parallel computing is often used on problems that cannot be solved by more conventional means (such as using a serial algorithm running on a PC or workstation).
Parallel computing can be utilized to perform numerous computations quickly and solve problems by making many processors work simultaneously on smaller subtasks or subsets of the data. Parallel computing is often used on problems that cannot be solved by more conventional means (such as using a serial algorithm running on a PC or workstation).
Most parallel applications require communication, called message passing, between subtask processes. Message passing is used for things such as loading or distributing initial data, exchanging data while running an algorithm, and collecting final results, which are usually distributed across the memories local to each processor at the end of execution.
In last month’s column, we focused on the basic concepts of parallel programming, including problem decomposition and granularity. We presented examples of program initiation and termination using both PVM (Parallel Virtual Machine) and MPI (Message Passing Interface), the two most popular message passing applications programming interfaces.
Nodes, Processors, and Processes
Computer architectures are becoming increasingly complex and hierarchical. Today, most computers larger than a desktop PC contain more than one processor. Beowulf clusters consist of individual computers called nodes. Most clusters today have nodes with multiple processors called symmetric multiprocessor (SMP) nodes. Although individual processors within a single node have their own private cache memory, they can access and share the main memory in that node. Message passing provides a mechanism for the exchange of…
Please log in to view this content.
Not Yet a Member?
Register with LinuxMagazine.com and get free access to the entire archive, including:
