The free sharing of ideas, successes, and failures around Linux clustering was, in part, responsible for the rapid growth of Linux clusters. But, where are all the HPC user groups now?
Back in the winter of 1998 a small group of people held a meeting in Santa Fe, New Mexico to discuss Linux Clustering. The meeting was sponsored by the Advanced Computing Laboratory at Los Alamos National Lab (LANL) and the group was intentionally kept small as the event was by invitation only. Not that there would have been large crowds in any case. At the time Linux or Beowulf clusters were often considered bit of a back room lark that would never amount to much. I mean, who in their right mind would build supercomputers from off-the-shelf hardware and some free operating system written by a graduate student in Finland?
We all know how the Linux cluster story turned out, but often the important sidebars go unmentioned. For example, the little meeting in Santa Fe mentioned above. Somehow I was invited and found myself sitting in a meeting with some scary looking guy named Jon ‘maddog’ Hall, a guy who seemed to know a lot about Ethernet named Don Becker, then there was this excitable guy named Bob Young from a small company called Red Hat.
There were others, like Dave Turek of IBM, Walt Ligon from Clemson, and of course the guy who organized it all, Pete Beckmann from LANL. To keep the discussion open and honest, there were no precedings or documents created at this meeting. There were some presentations, but mostly we talked. And, we talked some more. The discussions ranged from clusters to Linux, to TCP stacks, to cigars and cognac. (This was the birthplace of the Ligon/Eadline Cigar and Cognac Invitational Beowulf Gathering (LECCIBIG) by the way.)
The conversation had begun. It continued at, other Extreme Linux workshops, Birds of Feather sessions at the annual Supercomputing Show, tutorial and presentations at Linux events, and on and on. The Beowulf mailing list, which was sometimes lucky to have a post or two a week, was now full of interesting conversations.
The free sharing of ideas, successes, and failures was in part responsible for the rapid growth of Linux clusters. When ever I attended a Linux cluster event, I was always more interested in the side discussions than the formal presentations. Much like workplace gossip, I was intrigued with Beowulf scuttlebutt as it were. I still am, by the way.
Looking at the bigger picture it seems the power of groups and group discussions has garnered some attention. This past summer I read a book called The Wisdom of Crowds. In this book author James Surowiecki describes how a groups tend to get things right, even with incomplete data. He even cites the rise of Linux as an example of successful group dynamics. The right kind of large groups have been shown to out perform small groups of experts in many circumstances. Groups are good and open discussion is even better.
So, where are the HPC users groups?
Glad you asked! To date, the only two I have known about have been the BWBUG (Baltimore-Washington Beowulf User Group) and the BayBUG (Bay Area Beowulf Users Group) in San Francisco. The dearth of HPC user groups, and my penchant for discussion, got me thinking about my local area. My conclusion is that we need a HPC user group in the Northeast, and New York City seems like the place to start. Besides, I hear there are a lot of people there.
So, without further fanfare, I am pleased to announce the NYCA-HUG (New York City Area HPC Users Group). Our intention is to create a gathering place for HPC computer enthusiasts to talk about clusters and other such things. We will be holding hold monthly meetings, discuss topics of interest to group members, hear from invited guest speakers, and dish out our fair share of cluster scuttlebutt . To find out more, please visit or Web page. You can even join the mailing list, if you like.
For those in the Metropolitan area, we will be holding our first meeting on November 1, 2007 at 590 Madison Avenue (IBM Building) from 7:00 p.m. to 9:00 p.m. Our first topic will be New Directions in HPC. Come on by and catch up on the latest high tech scuttlebutt.