[Beowulf] MPI programming question: Interleaved MPI_Gatherv?

[Rob Ross]

Robert G. Brown wrote:
> On Thu, 3 Mar 2005, Joachim Worringen wrote:
> 
>>Michael Gauckler wrote:
>>
>>>I would like to gather the data from several processes. 
>>>Instead of the comonly used stride, I want to interleave 
>>>the data:
>>>
>>>Rank 0: AAAAA -> ABCDABCDABCDABCDABCD
>>>Rank 1: BBBBB ----^---^---^---^---^
>>>Rank 2: CCCCC -----^---^---^---^---^
>>>Rank 3: DDDDD ------^---^---^---^---^
>>>
>>>Since the stride of the receive type is indicated 
>>>in multpiles of its mpi_type, no interleaving is 
>>>possible (the smallest striping factor leads to 
>>>AAAAABBBBBBCCCCCDDDDD).
>>>
>>>Is there a way to achieve this behaviour in an 
>>>elegant way, as MPI_Gather promises it? Or do
>>>I need to do Send/Recv with self-aligned offsets?
> 
> 
> What about RMA-like commands?  MPI_Get in a loop?  Since that is
> controlled by the gatherer, one would presume that it preserves call
> order (although it is non-blocking).

I would hope that one would read the spec instead!  MPI_Get()s don't 
necessarily *do* anything until the corresponding synchronization call. 
  This allows the implementation to aggregate messages.  Call order (of 
the MPI_Get()s in an epoch) is ignored.

> Or of course there are always raw sockets... where you have complete
> control.  Depending on how critical it is that you preserve this strict
> interleaving order.
> 
>    rgb

No you don't!  You're just letting the kernel buffer things instead of 
the MPI implementation.  Plus, Michael's original concern was doing this 
in an elegant way, not explicitly controlling the ordering.

Joachim had some good options for MPI.

Regards,

Rob