[Beowulf] Using Autoparallel compilers or Multi-Threaded libraries with MPI

[richard.walsh at comcast.net]

-------------- Original message -------------- 
From: Håkon Bugge <Hakon.Bugge at scali.com> 

> I have a slightly different view. Hybrid 
> programming is used for performance reasons, but 
> only in cases where parallelization (to the same 
> level) is impossible/impractical using the pure 
> MPI mode, or the parallelization yields low 
> efficiency. So, if you're able to achieve your 
> performance with MPI, you probably will. But 
> there are cases where you cannot; a) the 
> "decomposition parallel efficiency" is not good 
> enough or b) the processes need a huge (shared) table. 

I think that what is being said here is that applications may be decomposible in some number of dimensions, but not so in all.  If the benefits in performance in locally managing the "unruly" dimensions are great enough, then a hybrid program may be worth the trouble.  I think that the number of real-world apps in this class is perhaps not large, or there would be more hybrid code. 
Another perhaps relavent alternative that will at some point be able to take on both the partionable and unpartionable extreme cases and everything in between are the PGAS language extensions (UPC and CAF).  Not yet at distributed-memory, performance-parity with well-coded MPI, but with, arguably, an intrinsic programmability advantage in LOC and in data structure coverage.  AMR codes tracking shedding vortices are inherently non-partionable (or in need of regular repartitioning).  Managing then in either MPI or OpenMP in a distributed memory environment is a chore.
And if you believe that ... ;-) ... then there is of course the "magic" of many-threaded latency hiding (can't say I am a true believer for the data intensive OZ of HPC).  Some would have you believe that a 32 thread, 8 core Niagara 2 (or perhaps a future design at some higher active thread to core ratio) can hide all your data latency events behind its active thread horizon.  
Maybe the key is to combine PGAS with many-threads ... mmm ... anyone doing this?
;-) 
rbw
-- 

"Making predictions is hard, especially about the future." 

Niels Bohr 

-- 

Richard Walsh 
Thrashing River Consulting-- 
5605 Alameda St. 
Shoreview, MN 55126 

Phone #: 612-382-4620
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20071203/a286e39b/attachment.html>