[Beowulf] Re: Re: Home beowulf - NIC latencies

[Robert G. Brown]

On Wed, 16 Feb 2005, Patrick Geoffray wrote:

> > This is however too much detail for this forum though, as most of the 
> > postings here discuss much more practical issues :)
> 
> I am bored with cooling questions. However, it's quite time consuming to 
> argue by email. I don't know how RGB can keep the distance :-)
> 
> Patrick
> 

I stuck a hairpin into an electrical socket at age 2 (an "enlightening"
experience I must say) and had a large rock fall on my head from a
height of almost a meter at age 8.

Since then, I hardly ever get bored with cooling questions, because I
cannot remember that they've been asked.  What were we talking about,
again?

Oh yeah, MPI and all that.

I've actually been enjoying reading the discussion and not
participating, since I'm a PVM kinda guy.  But SINCE my name was invoked
in vain, I'll make a single comment on the code quality issue, which is
that underlying the discussion of communication pattern, blocking vs
non-blocking, and directives is the fundamental scaling properties of
the code and algorithm itself.  So on the issue of whether MPI sucks
because the application sucks -- well, possibly, but it seems more
likely that the application sucks because its parallel scaling
properties (with the algorithm chosen) suck.

As to how "intelligent" the back end library should be at choosing
algorithm -- I would say the BASIC library should be atomic, elementary,
NOT algorithm level stuff.  A thin skin on top of raw networking calls
that provides the various things one always has to do oneself but not
much more.  Where one gets into trouble is where one uses a command that
has a complex structure that doesn't fit your code without realizing it,
and the reason you don't realize it is because all that detail is
hidden, and isn't even uniform in RELATIVE performance across varying
network hardware.

In other words, to make MPI do more, either make it do less (in the form
of commands that can be used to build "more" in a manner that is tuned
to application and hardware) or be prepared to REALLY make it SMART
behind the scenes.

This isn't just MPI, BTW.  PVM suffers from the same thing.  I honestly
think that both are limited tools in part BECAUSE they put too thick a
skin between the programmer and the network.  If you want real
performance and complete control over communication algorithm, you
probably have to use raw/low level networking commands, and write the
appropriate "collective" operations for your particular application and
hardware.

Of course nobody does this -- not portable and a PITA to
design/write/maintain.  Or perhaps a few people DO do this, but they're
programming gods.  And this isn't crazy, really.

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu