|
|
Building HPC Clusters
Here is your challenge. You have a need for speed, your current computing power is insufficient for the task at hand. You have some large number of calculations to perform, and very little time to achieve this goal. Can you solve this problem? Cost effectively? Quickly?
Some relief for cluster consternation.
As the new Intel Xeons hit the street, our resident HPC expert spends two weeks with sixteen cores.
The correct choice of I/O & communication interconnects can increase application performance by 2-3x. However, interconnects and their interaction with highly-parallel applications are often not well understood. Learn how the architecture of InfiniBand and Ethernet interconnects impact performance on a variety of applications.
Managing HPC Clusters
High Performance Computing (HPC) clusters are easier, and cheaper, than ever to put together. If you have an interest in pulling together your own cluster, or maybe you just want to understand more about cluster technology, it’s necessary to grok the differences between clusters and standard systems.
Tyan's PSC comes packed with forty cores, Infiniband, and much more inside the cube than at first meets the eye.
What’s stopping clusters from being useful tools?
While implementing and managing a powerful and complex cluster environment can seem like a daunting task, you can make your life much easier and your users more productive by sticking to a few simple rules. Here’s a guide.
If you need to monitor and manage such a configuration, try syslog-ng (syslog, next generation), a drop-in replacement for syslogd. syslog-ng provides more sophisticated log management capabilities and enables log transfers over the Internet.
| Top Stories
In part one, I introduced the two most popular HPC networking technologies -- Ethernet (GigE and 10GigE) and Infiniband. We also compared latency, bandwidth, and the N/2 performance of these technologies. While these numbers give a general feel for performance, there is no easy way to determine the actual performance of your application.
This week I'd like to take a look at the latest survey results. While the participation was not all that I wanted, it is enough to make sufficiently vague statements about the HPC cluster market.
In the last installment of our Parallel Platters series, Jeff Layton looks at the next generation of parallel file systems: Object Based File Systems.
Earlier this year, my wife and I decided to finish our basement. Great! I thought I could finally set up a real office. I had just one thing to do before my office would become a reality — deal with the ghosts of systems past.
I find Pablo Picasso's famous response on computers, "Computers are useless. They can only give you answers," interesting and provocative. I also believe there is a grain of truth in that statement — namely, that it's really important to ask the right questions of computers. I have a hard time with the useless part, however.
When comparing CPUs, you have to compare apples to apples. Doug Eadline compares the performance of AMD Opteron and Intel multicore processors to see which CPUs provide the best performance per core.
Continuing the discussion of cluster-ready filesystems with an overview of traditional parallel file systems, which allow clusters to contact multiple storage devices directly rather than communicating with storage through a gateway. All you ever wanted to know about parallel file systems, and then some.
Back in the good old days of single core processors, when HPC clustering was in its infancy, getting the application as close as possible to the hardware was very important. In many cases, it still is. Communication between nodes could take place through the operating system by using TCP/IP or outside the OS using a userspace zero-copy protocol. With the exception of pinning down memory, the userspace protocol totally removes the OS from the communication. The result of userspace communication is better application performance thanks to better latency and throughput.
A story floating around the Internet recently claims that the Storm email worm has created the world's largest supercomputer. I believe the genesis of this claim was this post on a security Web site. Are you amazed? Do we need James Bond to ferret out the evil villains? Hardly. Perhaps Austin Powers can handle this one.
As vendors strive for faster processors and denser systems, power and cooling has become a major issue for the HPC market.
Explicit parallel programming presents special challenges for software developers. Now a new group of languages are coming online to address the compounded problem of multi-core processors on high-performance clusters.
Every time I talk about multi-core, I seem to start out with something like "back in the day" or "when things were much simpler," or some such lament. Now prepare yourself for a stunning bit of insight. Cue music.
HPC cluster optimization is often simple. Avoiding assumptions is hard.
Clusters are getting larger, multi-core adoption plods along, and other findings from our recent HPC micropolls.
There I admit it. There are certain things that send me into long rants when it comes to High Performance Computing (HPC). (We'll skip the non-HPC issues for now). I'll bet you have issues as well. Those things that just bug you about the state of HPC clusters. Admit it, you do. There, don't you feel better?
|
|
|
|
 |
Podcast: Ted Ts'o InterviewIn this podcast, Ted Ts'o, the Linux Foundation's newly appointed Chief Platform Strategist, takes a few minutes to talk to Linux Magazine about his new role with the Linux Foundation, the status of Ext4, the Linux Standard Base, and more. |
|
Podcast: Interview with Ubuntu's Jono Bacon and Jorge Castro This week we spent some time talking to Ubuntu Community Manager Jono Bacon, and External Projects Developer Liaison Jorge Castro, about the Ubuntu community, Personal Package Archives, and where Linux is headed in 2008. |
| RSS |
|
Sponsored Links
|
