[Beowulf] MPI Implementations for SMP use

[Ed Hill]

On Fri, 2004-12-17 at 22:00 -0800, Greg Lindahl wrote:
> On Wed, Dec 15, 2004 at 08:56:05PM +0000, Eric Thibodeau wrote:
> 
> > Probably opening up another can of worms but, you might want to
> > seriously consider a hybrid (MPI + OpenMP or Pthreads) approach in
> > the case of SMP machines. Local exeution of OpenMP generated code is
> > considerably faster than the locally executed MPI
> > equivalent.
> 
> You would think that, but the nice thing about pure MPI is that
> locality is perfect. So in most cases, a pure MPI code beats a hybrid
> code. That's good, because hybrid programming is more complicated than
> straight MPI or straight OpenMP.
> 
> Most of the folks interested in hybrid models a few years ago have now
> given it up.


Hi Greg,

Do you have any references concerning hybrid pthreads+MPI vs. MPI-only
on clusters of SMP/NUMA systems?  I'm not at all trying to dispute your
claim!  I'd just like to learn more about the details.

I'm interested because our code (MITgcm.org) has the ability (at least,
theoretically) to do ptheads, MPI, or hybrids though people rarely use
pthreads and almost never try hybrid arrangements.  It seems that there
might be some very real benefits from the hybrid approach.

But thats just my intuition speaking!  Perhaps there are MPI
implementations that are competitive with threads since they can very
efficiently handle this "two-level" nature of communication: keeping it
local for processes on the same node while still *simultaneously* taking
care of the over-a-network bits for the rest.

Ed

-- 
Edward H. Hill III, PhD
office:  MIT Dept. of EAPS;  Rm 54-1424;  77 Massachusetts Ave.
             Cambridge, MA 02139-4307
emails:  eh3 at mit.edu                ed at eh3.com
URLs:    http://web.mit.edu/eh3/    http://eh3.com/
phone:   617-253-0098
fax:     617-253-4464