Fwd: [Beowulf] Re: Why is communication so expensive for very small messages

[Mark Hahn]

> zero, but MPI_Wtime() only has a resolution of 4ms.

why use MPI_Wtime?  I use:

#include <sys/time.h>
double gtod() {
     struct timeval tv;
     gettimeofday(&tv,0);
     return tv->tv_sec + 1e-6 * tv->tv_usec;
}

portable, high-resolution.  you might be thinking "oh, but MPI_Wtime() is
based on clock(3) or times(2), which have to be better because they are 
mandated by more standards and provide the tempting distinctions between 
wall, user and system times.  alas, the resolution or accuracy of such
accounting is never of much importance to the kernel, so has to be ignored.

in short, use wallclock time from gettimeofday...

if you want, I can send my rdtsc-based timing routines, but they need 
to be calibrated and you have to consider cpu binding on anything other
than single-core systems...

regards, mark hahn.