[Beowulf] MPI - time for packing, unpacking, creating a message...

[Nifty Tom Mitchell]

On Sat, May 23, 2009 at 12:43:57PM -0300, Bruno Coutinho wrote:
> 
>    If you are using Gigabit Ethernet with jumbo frames (9000 bytes for
>    example):
>    A will send 3 packets with 4000 bytes and
>    B will send one of 9000 bytes and one of 7000 bytes.
>    For the cpu B is better, because will generate one system call and A
>    will generate three and
>    as many high speed interconnects today need large packets to fully
>    utilize their bandwidth, I think that B should be faster.
>    But the only way to be sure is testing.
>    2009/5/18 <[1]tribur at vision.ee.ethz.ch>
> 
>      Hi all,
>      is there anyone who can tell me if A) or B) is probably faster?
>      A)
>      process 0 sends 3x500 elements, e.g. doubles, to 3 different
....
>      B)
>      process 0 sends 2000 elements to process 1 using

Not all MPI transports require a system call.  Some transports can see
hardware and move data without the overhead of a system call.  In such 
a case the issue is byte count more than anything else.

The next topic might be 'buffering'
   http://www.mpi-forum.org/docs/mpi-11-html/node40.html#Node40
Since this is a blocking send the status of the receive buffer is
important.  No blocking send can be finished without a valid destination 
buffer and reply-- this may depend more on the application MPI-communications 
and memory state then on the isolated code snips we see.   And for
the application the critical issue may not be the completion of the
send but delivery of the data to the application.  Since the blocking send
can be completed yet the application not 'see' the data.

Both isolated code snips move small enough chunks of data that it 
is unclear how well the transfer time can be measured.  For example
timing could be dominated by the page fault for MPI_send and not by 
the data transfer itself.

Worth repeating.
>    But the only way to be sure is testing.






-- 
	T o m  M i t c h e l l 
	Found me a new hat, now what?