[Beowulf] Re: Re: Home beowulf - NIC latencies

[Rob Ross]

On Mon, 14 Feb 2005, Ashley Pittman wrote:

> On Fri, 2005-02-11 at 20:47 -0600, Rob Ross wrote:
> > I agree that people using MPI_Isend() and related non-blocking operations 
> > are sometimes doing so because they would like to perform some 
> > computation while the communication progresses.  People also use these 
> > calls to initiate a collection of point-to-point operations before 
> > waiting, so that multiple communications may proceed in parallel.  The 
> > implementation has no way of really knowing which of these is the case.
> 
> Either of these reasons for using non-blocking sends is valid and both
> will benefit from low CPU use in the Send call.  Why would the
> implementation want to know the reason for using non-blocking sends?

If you used the non-blocking send to allow for overlapped communication, 
then you would like the implementation to play nicely.  In this case the 
user will compute and eventually call MPI_Test or MPI_Wait (or a flavor 
thereof).

If you used the non-blocking sends to post a bunch of communications that
you are going to then wait to complete, you probably don't care about the
CPU -- you just want the messaging done.  In this case the user will call 
MPI_Wait after posting everything it wants done.

One way the implementation *could* behave is to assume the user is trying
to overlap comm. and comp. until it sees an MPI_Wait, at which point it
could go into this theoretical "burn CPU to make things go faster" mode.  
That mode could, for example, tweak the interrupt coalescing on an 
ethernet NIC to process packets more quickly (I don't know off the top of 
my head if that would work or not; it's just an example).

All of this is moot of course unless the implementation actually has more
than one algorithm that it could employ...

Rob