[Beowulf] multi-threading vs. MPI

[Geoff Jacobs]

richard.walsh at comcast.net wrote:
>  
> 
>     -------------- Original message --------------
>     From: "Toon Knapen" <toon.knapen at gmail.com>
>      
>     How come there is almost unanimous agreement in the
>     beowulf-community while the rest is almost unanimous convinced of
>     the opposite ? Are we just tapping ourselves on the back or is MP
>     not sufficiently dissiminated or ... ?
>      
>     Mmm ... I think the answer to this is that the rest of world
>     (non-HPC world) is in a time
>     warp.  HPC went through its SMP-threads phase in the early-mid 1990s
>     with OpenMP, and then we needed more a more scalable approach
>     (MPI).  Now that multi-core and multi-socket has brought parallelism
>     to the rest of the Universe, SMP-based parallelism has had a
>     resurgence ... this has also naturally caused some in HPC to revisit
>     the question as nodes have fattened. 
>      
>     The allure of a programming model that is intuitive, expressive,
>     symbolically light-weight,
>     and provides a way to manage the latency variance across memory
>     partitions is irresistable.
>      
>     I kind of like the CAF extension to Fortran and the concept of
>     co-arrays.  The co-array is
>     and array of identical normal arrays, but one per active
>     image/process.  They are defined as such:
>      
>               real, dimension (N) [*] ::  X, Y
>      
>     If the program is run on 8 cores/processors/images the * becomes 8. 
>     8, 1D arrays of size
>     N are created on each processor. In any references to the locale
>     component of the co-array
>     (the image on the processor referencing it), you can drop the []s
>     ... all other references (remote)
>     must include it.  This is symbolically light, but reminds the
>     programmer of every costly non-
>     local reference with the presence of the []s in the assignment or
>     operation.  There is much
>     more to it than that of course, but as the performance gap between
>     carefully constructed
>     MPI applications and CAF compiled code shrinks I can see the later
>     gaining some traction
>     for purely programming elegance related reasons.  If you accept that
>     notion that most MPI
>     programs are written at a B- level in terms of efficiency then the
>     idea of gap closing may not
>     be so far fetched.  CAF is supposed to be include in the Fortran
>     2008 standard.
>      
>     rbw
>      
>     -- 
> 
>     "Making predictions is hard, especially about the future."
> 
>     Niels Bohr
> 
>     -- 
> 
>     Richard Walsh
>     Thrashing River Consulting--
>     5605 Alameda St.
>     Shoreview, MN 55126 

But isn't CAF (and UPC, and Titanium) implicitly message passing for a
Beowulf anyway? It's attractive because it simplifies the process and
might be able to optimize communication, but underneath everything it's
still message passing.

-- 
Geoffrey D. Jacobs

To have no errors
  would be life without meaning
  No struggle, no joy