Parallel File Systems: File Systems for HPC Clusters (Part Two)
Continuing the discussion of cluster-ready filesystems with an overview of traditional parallel file systems, which allow clusters to contact multiple storage devices directly rather than communicating with storage through a gateway. All you ever wanted to know about parallel file systems, and then some.
In part one of this series covered distributed file systems, focusing primarily on those using NFS as the protocol. The sole exception is parallel NFS (pNFS) (http://www.pnfs.com/), which could easily become the future parallel file system for clusters. However, it will take some time for it to be approved and for vendors to write the appropriate drivers.
In the meantime, parallel file systems are available for clusters now. This installment will touch on some of the major parallel file systems that use more traditional approaches to file systems and are available now. Once again, I can’t cover every parallel file system available, and I apologize if I don’t cover one that you are interested in or one that you use.
Parallel File Systems
Parallel file systems- what do they look like? Parallel file systems are a hotly discussed technology for clusters. They can provide lots of I/O for clusters if that’s what you truly need– and often, it is needed. I’m constantly surprised at how many people don’t know how I/O influences the performance of their applications.
They can also provide a centralized file system for your clusters, and in many cases you can link parallel file systems from geographically distinct sites so you can access data from other sites as though it was local. Centralized file systems can also ease a management burden and improve the scalability of your cluster storage. In addition, centralized file systems can also be used for diskless systems by providing a reasonable…
Please log in to view this content.
Not Yet a Member?
Register with LinuxMagazine.com and get free access to the entire archive, including:
