[Beowulf] multi-threading vs. MPI

[richard.walsh at comcast.net]

-------------- Original message -------------- 
From: "Toon Knapen" <toon.knapen at gmail.com> 


How come there is almost unanimous agreement in the beowulf-community while the rest is almost unanimous convinced of the opposite ? Are we just tapping ourselves on the back or is MP not sufficiently dissiminated or ... ? 

Mmm ... I think the answer to this is that the rest of world (non-HPC world) is in a time
warp.  HPC went through its SMP-threads phase in the early-mid 1990s with OpenMP, and then we needed more a more scalable approach (MPI).  Now that multi-core and multi-socket has brought parallelism to the rest of the Universe, SMP-based parallelism has had a resurgence ... this has also naturally caused some in HPC to revisit the question as nodes have fattened.  

The allure of a programming model that is intuitive, expressive, symbolically light-weight,
and provides a way to manage the latency variance across memory partitions is irresistable.

I kind of like the CAF extension to Fortran and the concept of co-arrays.  The co-array is
and array of identical normal arrays, but one per active image/process.  They are defined as such:

          real, dimension (N) [*] ::  X, Y

If the program is run on 8 cores/processors/images the * becomes 8.  8, 1D arrays of size
N are created on each processor. In any references to the locale component of the co-array
(the image on the processor referencing it), you can drop the []s ... all other references (remote)
must include it.  This is symbolically light, but reminds the programmer of every costly non-
local reference with the presence of the []s in the assignment or operation.  There is much
more to it than that of course, but as the performance gap between carefully constructed
MPI applications and CAF compiled code shrinks I can see the later gaining some traction
for purely programming elegance related reasons.  If you accept that notion that most MPI
programs are written at a B- level in terms of efficiency then the idea of gap closing may not
be so far fetched.  CAF is supposed to be include in the Fortran 2008 standard.

rbw

-- 

"Making predictions is hard, especially about the future." 

Niels Bohr 

-- 

Richard Walsh 
Thrashing River Consulting-- 
5605 Alameda St. 
Shoreview, MN 55126 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20071207/6ca9dd9b/attachment.html>
-------------- next part --------------
An embedded message was scrubbed...
From: "Toon Knapen" <toon.knapen at gmail.com>
Subject: [Beowulf] multi-threading vs. MPI
Date: Fri, 7 Dec 2007 20:07:32 +0000
Size: 721
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20071207/6ca9dd9b/attachment.mht>