[Beowulf] multi-threading vs. MPI

[Geoff Jacobs]

Douglas Eadline wrote:
> This is indeed the issue. Where to invest time?
> 
> My opinion, and it is only my opinion, is the following.
> Please share your own.
> 
> Threaded approaches do not scale across clusters. The memory
> architecture of multi-core is making nodes look more like
> small clusters i.e. memory is becoming more localized.
> As Don Becker mentioned in a recent post, efforts to program
> distributed memory like it were shared memory often end
> up looking like stylized message passing systems.
> 
> One other thing about messages. The problem of
> trying to optimize the compute to communication issue is
> easier than trying to optimize the compute to locality
> issue.
> 
> Therefore, if I were to start a new parallel project of some sort
> or parallelize an existing code, I would use MPI. Although
> OpenMP might get me up and running quicker, I would feel more
> comfortable with a problem cast in MPI.
> 
> I'm interested in others opinions on this because, I think it
> is an important issue for the general programing audience
> and not just us cluster geeks. The difference is we have had
> a lot more time and experience with this stuff.
> 
> --
> Doug

Please note that we are all speaking as developers on clusters. Although
this is a valuable niche in the market, it is still only a niche. Even
with workstations, systems still only come with a few cores. In this
regime, threads are still relevant from a performance standpoint.
Furthermore, threads (of whichever flavor) are a first class API in most
operating systems. Until Joe Sixpack can buy Windows ZX (or whatever)
with the new message passing system preloaded, the old system which is
preinstalled will still be the development target of most ISVs.

-- 
Geoffrey D. Jacobs