[Beowulf] MPI programming question: Interleaved MPI_Gatherv?
joachim at ccrl-nece.de
Thu Mar 3 01:14:40 PST 2005
Michael Gauckler wrote:
> Dear List,
> I would like to gather the data from several processes.
> Instead of the comonly used stride, I want to interleave
> the data:
> Rank 0: AAAAA -> ABCDABCDABCDABCDABCD
> Rank 1: BBBBB ----^---^---^---^---^
> Rank 2: CCCCC -----^---^---^---^---^
> Rank 3: DDDDD ------^---^---^---^---^
> Since the stride of the receive type is indicated
> in multpiles of its mpi_type, no interleaving is
> possible (the smallest striping factor leads to
> Is there a way to achieve this behaviour in an
> elegant way, as MPI_Gather promises it? Or do
> I need to do Send/Recv with self-aligned offsets?
Actually, I don't see an 'elegant' way to do this, either. The decision
between multiple MPI_Gatherv() calls and a Irecv/Send/Waitall construct
depends on the quality of the MPI implementation you use (MPI_Gatherv
can be optimized well for small amounts of data), the characteristics of
you interconnect (high latency gives more room for optimization) and the
number of processes you use. For small process numbers, you wont see
much of a difference anyway.
You could also try to gather all data on the root in separate buffers,
and then let this process send/recv to itself using the proper datatypes.
Finally, if this communication is not a significant part of your
runtime, you shouldn't spend much time optimizing it anyway.
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de
More information about the Beowulf