The Scent of the HPC Market

This week I'd like to take a look at the latest survey results. While the participation was not all that I wanted, it is enough to make sufficiently vague statements about the HPC cluster market.

This week I’d like to take a look at the latest survey results. While the participation was not all that I wanted, it is enough to make sufficiently vague statements about the HPC cluster market.

Whenever I talk about Web surveys I always throw in a proviso that these are not well-designed, scientific surveys performed by more sophisticated marketing organizations. Web surveys are more like asking the person on the street what they thought of a movie. When I look at the survey results, think of it as a “scent” of the market and maybe suggest some area that bear watching. But, enough handwaving. Let’s look at some results.

The question that garnered the greatest response (141 votes) was “What will limit your use of HPC Clusters in the next 12 months?” In other words, what is getting in your way? The one choice that surprised me was that sixteen percent of the people said “nothing.” You go people! Buy, build, and compute. On the other hand, 25% said that lack of skilled people was the biggest hold-back.

Software and hardware, space and power each weighted in at about 16%. Interestingly, cluster management issues were only of concern to 10% of the respondents. In previous surveys, cluster management issues seemed a bit more prevalent. This is a good trend, if it is true.

The people issue has been with us since the beginning. This result is supported by the fact that I am often asked if I “know any one?” who and would like a job working with HPC clusters. Unfortunately my answer is always, “Not really, everyone I know is busy and seemingly happily engaged in their current job.”

In this survey, I wanted to drill down and ask what kind of people were actually needed. The results (107 votes) were application specialists (31%), system administrators (27%), programmers (25%), none (11%), project managers (3%), hardware specialists (3%). The “none” result corresponds well with the “Nothing” result of the previous question. The top three needs are of course about software. The even spread across the top three responses indicates we need cluster knowledge across the board, as it were.

It seems the whole cluster thing snuck up on the market, and probably spread into markets that could not afford HPC in the past. The result is people who really understand this stuff (they have what I like to call “cluster scar tissue”) are hard to find. We need to fix this situation and things like Cluster Training at Georgetown University are helpful. (Full disclosure: I teach the fun filled Intermediate Beowulf Administration and Optimization course.)

Moving on to the other questions, I’m going to take a very cautious approach as the number of responses are quite low. Let’s take a look at the Infrastructure and the Parallel File Systems questions. Power and cooling seem to be the big issue for 75% of the 22 respondents. The rest considered cluster footprints to be their biggest concern. The parallel file system question had low participation as well (22 votes), but showed that parallel file systems were becoming more important for over half the respondents (55%), critical to project success for 36% of the respondents, and really not an issue for only 9% (NFS works fine).

Finally, there were two questions about hardware and software struggles. In terms of hardware, the biggest issues were compute node cost/performance (29%), interconnect cost/performance (24%), vendors don’t understand my needs (21%). Choosing the best hardware (13%) and component failures (13%) rounded out the 38 respondents. Other than noting an opportunity for vendors, I’m not putting much stock in these results, since there were five possible answers and only 38 respondents. A few more votes could easily change the result.

Turning to software, the results were as expected even with 22 respondents. The number one HPC software issue seems to be efficient use of multi-core processors (55%). Other issues, came in pretty even with application costs and cluster management tools, both at 14%, and lack of software tools and applications (each at 9%). The only thing that surprised me was the paltry number of votes for software tools, but the low number of respondents limits any real conclusions.

You can check out the survey results for yourself (after you vote) at Today’s HPC Clusters. We will leave current surveys up for about a week at which point a new batch of HPC questions will appear. Then it’s your turn. I don’t need to remind anyone of the importance of voting in the coming months. Outcomes that will influence the nation’s and even the world’s future will be determined. Of course, I’m talking about American Idol.

Fatal error: Call to undefined function aa_author_bios() in /opt/apache/dms/b2b/linux-mag.com/site/www/htdocs/wp-content/themes/linuxmag/single.php on line 62