[Beowulf] Why is communication so expensive for very small messages?
j.boyle at manchester.ac.uk
Tue Apr 24 08:54:49 PDT 2007
I apologise if this is a naive question, but I'm new to this world of
I'm using C++/mpi, to get a feel for communication costs I ran tests using
mpptest and my own programs.
For 2 processor blocking calls, mpptest indicates a latency of about 30
However when I measure communication times in my own program using a loop as
start = MPI_Wtime();
for (unsigned t=1; t<=5000; t++)
MPI_Send(data, size, MPI_INT, 1, tag, MPI_COMM_WORLD);
MPI_Recv(data, size, MPI_INT, 0, tag, MPI_COMM_WORLD, &status);
end = MPI_Wtime();
for size>=4, I get a latency of about 30 microseconds as expected, however for
size<4, communication costs increase massively, and latency now appears to be
Firstly, I assume this isn't normal?
Secondly, can anyone suggest what's going on, or where I can go for more
We're using mpich.
Processors are Intel(R) Xeon(TM) CPU 3.60GHz.
Interconnects are Dell PowerConnect 5324 24-port gigabit switches.
More information about the Beowulf