Tell me: What is the biggest issue facing cluster computing today?
I certainly cannot say. Sure, I have my own opinion and have a long list of things I want to see happen, but what’s important to me may not be of critical importance to you.
Assessing needs — what venture capitalists love to call “finding pain points” — is usually the job of marketing: Discover what customers want (what’s” ailing” them), build it, and then sell it. The problem with clusters, though, is that assessing market needs is a hard thing to do. Because clusters reach customers through a variety of channels, it’s hard to find the customers and ask them what hurts. At the recent Supercomputing 2005 conference in Seattle, IDC coined the term “dark clusters”. Like the “dark matter” proposed by some physicists, dark clusters cannot be seen or measured using conventional methods, yet their existence has an effect, nonetheless.
Enumerating needs is one problem, but a larger problem is one of identity. How is the cluster community going to grow and flourish if it can’t answer two simple questions: “Where are we?” and “Where we are going?” I constantly ask myself these questions and I find there’s no easy answer, yet clear conception and vision is vitally important to the vitality of the existing community and critically important if the community is to expand — that is, if clusters are to find new customers.
This issue of identity has been bugging me and some other people for over a year now. Neither they nor I can convey an answer, but we have an invitation. We’d like you to help us create a road map. The cluster community (including new users and vendors) will be served quite well by establishing a road map that illuminates the history of cluster computing and elucidates its future. We’d like you to participate in the The Cluster Agenda Initiative.
We Need an Agenda
Jeff Layton, Glen Otero, Dan Stanzione, and I started The Cluster Agenda Initiative (http://agenda.clustermonkey.net
). with the goal to produce a freely-available document titled The Cluster Agenda
to inform and guide existing and new members of the cluster community. The Cluster Agenda
isn’t an effort to introduce standards, critique products, or pick winners. Instead, it’s intended to be a living, breathing document that connects the many loose dots of the HPC cluster community.
And, thanks to that modern wonder called the Internet, we believe the Agenda can become a reality without too much effort.
One onus of the Agenda
is to identify challenges. To collect feedback, a survey that runs from July to December of each year will elicit responses from the community. Once each survey is complete, we’ll post the results and invite the community to comment. For those interested, the current survey — the 2005/2006 Why Do Clusters Suck?
survey — is available at http://agenda.clustermonkey.net/agendaform.html
. Community members are also invited to contribute to the road map. To facilitate collaboration, we’ve chosen to use a Wiki format (similar to Wikipedia
The milestones for the Cluster Agenda Initiative are pretty simple. We can put a check mark next to the December 2005 launch, and the next milestone is August 2006, when we hope to have have a draft version of The Cluster Agenda 2007. In subsequent years, Agenda 2008 will be prepared in 2007 and include updates made to the Agenda’s Wiki. As with all things HPC, we’ve timed our first release for November 2006 at Supercomputing 2006 in Tampa, Florida. We plan on releasing the Agenda in both print and digital form. (Why a “dead tree” version? Management simply responds better to hard copy. Call it “old school”, but if it’s in print, it has to be true, right?)
So, What Do I get Out of This?
Of course, I know what you are thinking: I’m going to go look at this site and see what is in it for me. Hopefully, more free stuff on the Internet. Yes, that’s the intention, but like most free things, there is a cost. To make it useful to you, someone else needs to contribute. Now, re-read the last sentence and interchange the words “you” and “someone else.” You get the idea. Give a little, get a lot.
I also want to emphasize that the Agenda is not an attempt to rewrite the history of cluster computing. It’s just a road map. If there’s a web page, resource group, or publication that addresses a certain area of cluster computing, simply take ten minutes and drop a link and comment in the Wiki. The Internet will take care of the rest. If there is a topic area that no one seems to have covered anywhere, then have at it. You are a pioneer after all.
We are using a Creative Commons Attribution, Non Commercial, ShareAlike 2.5 Common Deed, meaning that you can copy, share, edit, and reuse the Wiki content provided that a suitable attribution accompanies the content and you don’t charge money for it.
If you’re a seasoned cluster geek, I invite you to take a few moments and contribute your wisdom to the cause. I am continually astounded by the level and breadth of work with HPC clusters. The problem is I usually stumble upon it. I believe that if we catalog our efforts, we will be equally astounded by all of the information that’s available.
Your benefit is a stronger market, a bigger community, and more resources for you and your peers to access. If your pain is the same as my pain, then perhaps we can cooperate and help each other move forward. The alternatives include a community of half baked isolated efforts that narrowly address issues facing the community. With a little effort, we can do better. Thus, a project is born.
A Leg Up for Newbies
If you know nothing about HPC clusters, and you have made it this far into the column, welcome. I feel your pain. Finding good information about clusters isn’t an easy task. There are literally ten to twenty important decisions to make before using a cluster and finding answers or guidance is difficult.
In addition, if you take a look around this market and community and say, “HPC clustering a basically a bunch of disjoint efforts to use piles of servers for egghead science projects,” I would tend to agree with you. If you then walked away and told me to get back to you when things were a bit more stable and usable, I would certainly understand. But before you leave, I’d like to invite you to take a minute or two and answer a few questions in the survey and go the Agenda and enter a question (perhaps it is already answered). We can’t fix things if we don’t know what’s broken. And, check back from time to time. The Agenda is an expanding road map, after all.
A Gold Mine For Vendors
If you’re a vendor, you may be thinking, “Great! Another website to tell my story and put my link”. You can do that, but there is more gold than you might first guess.
In case you didn’t know, the growth of cluster computing has been nurtured by a strong community of designers, users, and vendors. Current cluster technology, however, is not without it’s challenges. Issues such as best practices, standards, benchmarking, education, and third-party software are often cited as “hold backs” to the continued growth of the market and increased sales. Yes, I said increased sales. When asked by potential customers about a specific area of HPC clustering, imagine having a resource to provide good answers and boost your customers comfort level.
If you’re still not convinced, then consider the The Cluster Agenda as a ongoing worldwide focus group for the HPC cluster market.
Vendors can help in two ways. First, contribute to the Agenda. If you have some ideas, concerns, best practices, or questions, add them to the Wiki. Second, since the Agenda is going to help make you more money, consider supporting the cause. Information about how you can support the Agenda is on the web site. And, yes, supporters get some nice perks.
We expect The Cluster Agenda to be the “go to document” for the entire cluster community. The Agenda is a great way for you to get in front of customers and show that you really understand the market and community.
Your Call to Action
Your next step is to visit the Agenda site and take the survey. When you’re done with the survey, go to the Agenda and contribute, comment, or ask a question. It couldn’t be any easier. Everyone is invited and everyone has something valuable to contribute.
If you’re still not motivated and remain convinced that someone else is going to chop the wood and carry the water, think about what you’re going to do when your colleague or customer is looking at all those nice glossy Microsoft Windows Compute Cluster Server brochures. What will you have to show them?
I feel your pain.