dcsimg

Emailing HPC

Email is not unlike MPI. The similarities may help non-geeks understand parallel computers a little better.

Parallel computing seems like a new concept to many people. In particular, multi-core now requires those that program for a living to grasp the basic ideas behind parallel computing. The concept of “parallel” should not be a surprise to anyone, however. The world is a very parallel place. We (humans) are social animals and thus must work (progress) together to solve many common problems, which is a pretty good definition of parallel computing.

Most news stories always provide a quote that relates the speed of a new parallel computer to a single processor. (e.g. “If this program were run on a single processor, it would take 100 years to finish.”) Sounds like breakthrough work, but no one really mentions the building that holds the cluster would take one person 50 years to build. Those types of statements seem silly. So why do parallel computers seem so exotic?

Working together seems natural and obvious. Indeed, parallel computing has been around for a while, but not in the mainstream. When venturing into parallel computing, there are concepts that are very familiar to any construction crew (like those building a new computer center); Amdahl’s Law of parallel computing is one example. It basically says you can’t go faster than the slowest single step. Anyone who has worked on a team project understands this. Message passing is another concept with which everyone is familiar — probably more than they know. I am thinking specifically of email which is probably the largest parallel human/silicon computer on the planet. Think about it. There are bidirectional messages connecting meat bag processors (people). Not only are commands sent and acknowledged, but data are also sent over this channel. For instance, consider a common email conversation:

Taylor sends to Lauren: Add your results (a command) to the attached document (the data)
Laure responds: Okay
Lauren sends to Taylor: The new version (a command) is attached (the data)
Taylor responds: Okay

Email also works as a universal transport mechanism. Pictures, documents, spreadsheets, music, all are sent via email. Actually with the right client it can be a very smart mechanism because it will often associate an application with the data payload.

The similarities to a typical HPC program are interesting. A large amount of collaborative (parallel) work is done via email. For instance, a document is sent to a group of people for processing (MPI_BCAST). They each complete their part and send it back to the team leader (MPI_SEND) who then does a reduction of the data. There may also be some two-way communication with various individuals (MPI_SEND/MPI_REC).

Like MPI, Email is “sent” when it is copied from the user to the host for transport. There is no “blocking” sends or receives or barriers, but these kinds of things could be worked out by the human endpoint processors. MPI requires a more explicitly send/receive structure, where sends and receives need to be matched. Email is totally asynchronous.

The MPI_COM_WORLD equivalent in email is literally the whole world, and therein lies a problem because a message will be accepted from anywhere — and thus SPAM. In general, MPI has a defined subset of endpoints, but unexpected messages can still be received if the endpoint is waiting for input. Email is in a constant receive loop.

Similar to an MPI program, data movement can be optimized by observing how it is used. For example, it is not uncommon for someone to create excessive email traffic by sending a large video file to all their friends. This type of broadcast is allowed in the email protocol. By placing the video in one location and broadcasting a short pointer to the video (e.g. posting on youtube and sending the URL to your friends), the endpoints can then use this data as they seem fit. A similar method is the use of Google Docs. A central copy (global data) is located in one location and pointers to the data are sent to the endpoints (collaborators). The endpoints can download some, none, or all of the global data. Similar concepts are used in parallel programming where optimal data movement can improve performance.

While I am not advocating “email based HPC,” I do think that it may be possible to build a simple generalized protocol on top of email that allows one to “play” and learn about parallel computing. For instance, a participating client (like your desktop) could handle “MPI-Mail” from other machines. If it were a matrix multiplication application and your client had the right handler program, it would just “know” what to do with it, process it and send a result email back to the sender (or another system designated by the sender). The human never sees the email, it is all done in the background.

Similar to BOINC, there could be a general framework for applications. Those that are developed using “MPI-Mail” can be run on “willing” clients. Of course, I glossed over a pile of things that would need to be addressed, but in principle it would make an interesting Senior project.

Finally, what I just described could also be thought of as “bot-net” programming language, so keeping it as a local academic exercise may be the way to go. In any case, the next time someone asks you, “What is MPI and parallel computing?”, just reply, “Email for computer processors, without spam.”

Comments on "Emailing HPC"

linus_cal

Who is this article intended for Mr. Deadline?
It start out with “those who program for a living” a.k.a professional software developers. If you have hired a developer that needs to be explained parallel programming via concepts of email, I think you need to improve your hiring criterion.

This can on the other hand, be used to explain things to kids in high-school.

Reply
eshneto

This is by far the worst article I’ve seen in this site. Not suitable for non-programmers (WTF is MPI?) neither for any decent programmer.

Reply
    linus_cal

    @eshneto MPI is Message Passing Interface. It is a distributed (on a cluster) computing middleware popular in scientific computing.

    Reply

Alright, person is as procesor and message is as email. Working in parallel, waiting for data or actions, doing tasks…

Reply

So you _can_ put 9 women together to give birth to a baby in 1 month after all. But all 9 of them will have to communicate via e-mail. Also knowledge of MPI is desirable, but not required.

Reply

I was waiting for such an article and I have gained some useful information from this site.
dissertation proposal writing paper

Reply

What is success? I think it is a mixture of having a flair for the thing that you are doing; knowing that it is not enough, that you have got to have hard work and a certain sense of purpose.
dissertation writers uk

Reply

dissertation writers uk
What is success? I think it is a mixture of having a flair for the thing that you are doing; knowing that it is not enough, that you have got to have hard work and a certain sense of purpose.

Reply

Genius Scientist – Achieving the Impossible

Dear Sir or Madam,

I would like to present “Achieving the Impossible”, a short story that can inspire other people; the story
presents how is possible to reach world-class level in a top research field and make a discovery, without having
prior training in the field, without speaking with anyone or getting help from anyone and almost without any money.

My name is Ovidiu Anghelidi and I made 3 discoveries:
1. I solved a complex molecular biology problem in neuroscience
2. I found an unbreakable encryption algoritm in computer science
3. I found a new source of energy in physics

I also ran the world’s largest brain simulation, see the following Discovery Channel article:
http://www.inventikon.com/articles/discovery.html

For additional information please see the following page:
http://www.inventikon.com/ovidiu/index.php

Thank you.

Sincerelly,
Ovidiu

Email: oanghelidi@inventikon.com
Phone: 1-647-701-8803

Reply

Cosmopolitan Announcement – gucci Believed An Absolute Must In these days dr dre mini bluetooth speaker http://www.zjubrc.com/beats-by-dre-mini-speakers-Monster-1/

Reply

Scene Report : gucci Described as An Essential Right now ???? ??????? ??? http://www.werbekatalog.com/chanel-Japan-123/

Reply

Matematyka wokó? nas sprawdzian do pobrania

Reply

hi, I am final year student from walchand college of engineering,sangli.I have my megaproject on “Publish-subscribed systems using mpi cluster in linux”.for that i implement cluster in ubuntu 12.04 with mpich2.now to proceed with project,what will be the next step?how to implement database organization in cluster?
Plz help.

Reply

Thank you for the good writeup. It if truth be told used to
be a enjoyment account it. Glance advanced to far brought agreeable from you!

However, how could we communicate?

Reply

Hey,

Stumbled upon your blog on Google and you definitely deserve more visitors!

Shared your blog on Facebook if that helps :)

Jared

Check out my site if you have time, maybe even share it on Facebook:
gta 5 cheats

Reply

Hey! I stumbled your blog on Yahoo. I am posting here to find
out what theme you’re using on this web page, I would like to use the theme that you are using
so I can put it on my website (pirater un compte facebook).

Thank you.

Reply

Although internet sites we backlink to below are considerably not associated to ours, we really feel they are actually really worth a go by, so have a look.

Reply

Hey very nice website!! Man .. Beautiful .. Wonderful .. I will bookmark your web site and take the feeds additionally…I am glad to seek out numerous useful info right here within the post, we want work out more strategies on this regard, thanks for sharing. . . . . .

Reply