Ever since then I have pondered how we solve the problem of managing parallel complexity in large scale programs. Much of the difficulty lies in the nature of specifying what constitutes a correct program. If the specification is too high level, you may get what you asked for but not what you want. But the more detail you supply, the more it becomes like a programming language with all its associated problems and less like a correctness specification.

About the most promising approach I have seen so far is the algorithmic skeleton. My first exposure to that was Murray Cole’s book “Algorithmic Skeletons”. Extending the idea to parallel skeletons is not a far stretch and seems to strike a good balance between high-level specification and efficiency. I’m sure such work is already under way. High level operators are already supported in MPI in the form of reduce, allreduce and other functions. They’re ugly syntactically because they are MPI, but the concepts are there.

While declaring the ultimate demise of computing due to high levels of parallelism does make for good entertainment, all I would really expect to see is the price of computation fall as parallel resources increase. When that happens we may not have single applications that can span significant fractions of the available compute power, say a million cores in a cluster. But virtually free compute power won’t be left idle, either. It will instead be used for lower valued tasks.

Douglas Pase, PhD

-Michael Wolfe (Hi Dr. Pase)

That said, I suppose that I fear change. My mind was honed pouring over assembly programmes, optimising them for size and speed.

If I could shave a few instructions, I might take another block (half K) off the image size, a significant consideration when the entire storage was one 35 MB washing machine.

While the “real programme” has been obscured since creation of the first compiler, I find the idea of “programming” in “PDL” to be truly alien, and a bit freakish.

I’d draw a parallel to modern electronics (especially digital) in which there is no fixed, discrete design.

While ‘twould be truly miraculous to be able to ask my computer to create an ad hoc application based upon my whim of the moment, a part of me will always long for the source listing.

We will have lost something important. We will have also gained more than we lost, but there will still be the ache of amputation.

Ah well, the tyrannosaur was likely no fan of progress.

Cheers,

Russ Bixby, geek of the plains

“What’s the bandwidth of localhost?”

Not to mention my kid’s game machines. 8+1 core Cell processors aren’t going to cut it for very long, and my last perusal of the Stanford Folding@Home READMEs hinted that the graphics processors were scaling core count through the roof and that they were quite happily making use of them.

Another prediction: just as home video became more real and hi-res cameras are now $449 commodities, the Linux and FreeBSD clusters that do animation and architectural rendering will scale to the deskside within the next 5 years.

Sure, these are not going to sell gazillions of PCs or MPPs in the next two weeks, but we will see an explosion of visualization-enabled reality simulations that will start showing up next to desktops and gaming joysticks (hopefully) near you Real Soon Now.

I think back to my first bare board 8086 with 2kx16 of RAM and a similar amount of UV EPROM, and I start to giggle insanely when I realize that the 2GB USB key I toss in my pocket has more CPU juice than it did.

None of which is meant to disagree with anything said. I think the most cogent point you both make is that money people have to see value in the same way that they grokked and then bought spreadsheets on PCs. Having a great analysis won’t save or make money until some bright bulb figures out what to do with it besides play.

– Don Wilde
no egg on my mortarboard, but I do work
for a rather large commodity PC maker

I do say that what everybody uses is MPI (mainly) and OpenMP (some). I just don’t consider those languages. As you say, they’re APIs.

One thing I do want to bring up: When I say “killer app” I think you understand but need to emphasize that it must be a client app, on the grounds that only clients have the volumes to justify the fabs. (See my “Clarifying the Black Hole” post.) There are many parallel server apps; no problem there. (See “IT Departments Should NOT fear Multicore” and elsewhere), But server volumes aren’t even visible next to clients.

Right now, it looks like those volumes are going to come from cell phones and other mobile applications, coming up from the bottom like PCs rose to meet workstations, with everybody stopping in the same place. I think I find this depressing. But a 32-way single-chip client is something I find, well, closer to terrifying.

- Greg Pfister