dcsimg

Concurrent and Parallel Are Not The Same

Pick one: portability or efficiency. Neither is guaranteed when writing explicit parallel code

In case you did not have a chance to read the column from last week, I am taking my yearly vacation at the Jersey Shore. Please refrain from the jokes, lest I pull out the Bruce Springsteen trump card. I try to spend two continuous weeks with family and friends each year. I have found that one week is just too short. I need two weeks. The first week is used to try and forget about all the stuff I did not get done before I threw the laptop in the car and say “let’s go.” The second week is used to try and remember and organize all the stuff I have to do when I get back. My plan usually breaks down somewhere around 10 AM my first day back to work.

This year I had a bunch of writing to do (including this column), so it was kind of a working vacation. Not to worry, I’ll have my feet in the Atlantic Ocean in few hours. In any case, my dilemma is as follows. Write an insightful column quickly and get to the beach. It may surprise some readers, but I do like to research some of the topics I write about. At a minimum, I like to include enough URLs so that if you actually want to investigate a topic further, more information is just a click away. As an aside, I am constantly amazed at how much content on the web has absolutely no external links to supporting material. I thought that was the whole idea. I mean how hard is it to add a Wikipedia link to a discussion of Clos Networks or some other networking technology.

Back to my dilemma. What can talk about that will get me to beach before the water ice guy packs up for day? Although, I don’t to like to rehash things I have written about in the past, I will be making a an exception this week. Not necessarily because it is easy, but because I think some messages need reinforcing. Therefore, all I have to decide is what message I should I hammer home on this July morning.

The answer is simple — understanding the difference between concurrent and parallel. I believe these two terms are often used interchangeably while, in my opinion, they are represent two different concepts.

Let’s start with concurrency. A concurrent program or algorithm is one where operations can occur at the same time. For instance, a simple integration, where numbers are summed over an interval. The interval can be broken into many concurrent sums of smaller sub-intervals. As I like to say, concurrency is a property of the program. Parallel execution is when the concurrent parts are executed at the same time on separate processors. The distinction is subtle, but important. And, parallel execution is a property of the machine, not the program.

If execution efficiency is important (i.e. you want things to go faster by adding more cores), then the question you need to ask is “If I run everything that is concurrent in parallel, will my code run faster?” If the answer were “yes” then we would not be having this discussion. And, since the answer, is “no”, then the question is “What should run in parallel?” which is obviously, the portions of code that lower execution time.

This decision is one of the reasons cluster parallel computing is hard. It really does depend on the machine. Take our integration case. If the integration interval is small, then breaking it up into small sub-intervals and sending them out to other nodes will result in extending the execution time of the program due to parallel overhead. If the integration interval is huge, then parallel execution may make sense. Because parallel overhead can vary from cluster to cluster, there is no easy way to predict overhead beforehand. (i.e. The parallel overhead is larger for GigE vs InfiniBand when sending small packets.)

The same applies to multi-core. The overhead for thread communication is lower, but there is still overhead (see my HPC Hopscotch for background on SMP memory). There is no free lunch — everyone has to deal with overhead.

In summary, the point I want to make is this, Concurrency is a property of the program and parallel execution is a property of the machine. What concurrent parts should and should not be executed in parallel can only be answered when the exact hardware is known. Which I might like to add leads to the most unhappy conclusion when dealing with explicit parallel programming, There is no guarantee of both efficiency and portability with explicit parallel programs. Yes, I know, a sad state of affairs. I’ll let you wrestle with that for a while, in the mean time, I’m going to the beach.

Comments on "Concurrent and Parallel Are Not The Same"

Usually posts some quite fascinating stuff like this. If you?re new to this site.

Just beneath, are several totally not associated internet sites to ours, having said that, they’re certainly worth going over.

Wonderful story, reckoned we could combine some unrelated data, nevertheless actually really worth taking a search, whoa did a single study about Mid East has got more problerms as well.

You’ve made some decent points there. I checked on the web for additional information in regards to the issue
and located the majority of people goes as well as your opinion of this website.

My page :: IsabelVCosio

Here are some links to websites that we link to since we believe they are worth visiting.

That will be the end of this post. Here you?ll obtain some sites that we assume you?ll appreciate, just click the hyperlinks.

That is the end of this article. Right here you?ll come across some websites that we assume you will enjoy, just click the links.

Although sites we backlink to beneath are considerably not related to ours, we really feel they may be really really worth a go by way of, so possess a look.

That will be the end of this write-up. Right here you?ll uncover some websites that we assume you?ll enjoy, just click the links.

We came across a cool internet site that you could delight in. Take a appear in case you want.

Please take a look at the internet sites we stick to, including this a single, as it represents our picks from the web.

Although websites we backlink to beneath are considerably not related to ours, we really feel they are actually really worth a go by, so have a look.

Every after inside a although we choose blogs that we read. Listed beneath would be the latest web pages that we pick.

Although internet websites we backlink to below are considerably not connected to ours, we really feel they are actually really worth a go via, so possess a look.

Check beneath, are some totally unrelated internet websites to ours, having said that, they are most trustworthy sources that we use.

Although internet sites we backlink to below are considerably not related to ours, we feel they’re essentially really worth a go by means of, so have a look.

Very few internet sites that take place to become comprehensive beneath, from our point of view are undoubtedly very well really worth checking out.

Below you?ll come across the link to some web pages that we assume you ought to visit.

Check below, are some totally unrelated web-sites to ours, even so, they’re most trustworthy sources that we use.

The information and facts talked about inside the article are a few of the ideal offered.

Wonderful story, reckoned we could combine a couple of unrelated information, nevertheless truly really worth taking a search, whoa did one find out about Mid East has got more problerms also.

That could be the finish of this article. Right here you?ll locate some internet sites that we feel you?ll appreciate, just click the hyperlinks.

Please stop by the web sites we stick to, which includes this a single, because it represents our picks from the web.

Here are a few of the web sites we suggest for our visitors.

The time to study or go to the subject material or websites we have linked to beneath.

Always a large fan of linking to bloggers that I appreciate but do not get a whole lot of link appreciate from.

Here are some of the sites we recommend for our visitors.

Please visit the websites we comply with, like this one particular, because it represents our picks in the web.

Very few sites that take place to be in depth beneath, from our point of view are undoubtedly effectively worth checking out.

Just beneath, are numerous entirely not related internet sites to ours, even so, they are certainly worth going over.

Wonderful story, reckoned we could combine a number of unrelated data, nonetheless definitely worth taking a search, whoa did 1 find out about Mid East has got much more problerms at the same time.

Usually posts some pretty fascinating stuff like this. If you are new to this site.

The information and facts mentioned in the report are a few of the best available.

Just beneath, are quite a few entirely not associated web sites to ours, however, they may be surely really worth going over.

We prefer to honor lots of other world wide web web sites around the net, even when they aren?t linked to us, by linking to them. Below are some webpages worth checking out.

Always a massive fan of linking to bloggers that I like but do not get quite a bit of link adore from.

Always a big fan of linking to bloggers that I adore but do not get quite a bit of link like from.

1Nz52a zrlatsbbuzwx, [url=http://fafrvbesyrtl.com/]fafrvbesyrtl[/url], [link=http://isvqqcxrpsgp.com/]isvqqcxrpsgp[/link], http://cnnkuefiiohq.com/

Here is a superb Weblog You might Come across Fascinating that we encourage you to visit.

Here is a superb Weblog You might Locate Exciting that we encourage you to visit.

The information mentioned within the report are a number of the best available.

Although websites we backlink to below are considerably not related to ours, we feel they are in fact really worth a go through, so possess a look.

Here are some hyperlinks to web-sites that we link to due to the fact we consider they’re worth visiting.

Always a huge fan of linking to bloggers that I adore but really don’t get a lot of link love from.

Check beneath, are some totally unrelated web-sites to ours, having said that, they’re most trustworthy sources that we use.

Always a massive fan of linking to bloggers that I appreciate but don?t get a lot of link appreciate from.

Check below, are some entirely unrelated web sites to ours, on the other hand, they are most trustworthy sources that we use.

I like this blog its a master peace ! Glad I found this on google .

Very few web sites that take place to be detailed beneath, from our point of view are undoubtedly effectively worth checking out.

One of our visitors lately suggested the following website.

Leave a Reply