[Beowulf] Home beowulf - NIC latencies

[Patrick Geoffray]

Hi Joachim,

Joachim Worringen wrote:
> Patrick Geoffray wrote:
> 
>> Seriously, here are MPI latencies with MX on F cards on Opteron 
>> (PCI-X), that includes fibers and a switch in the middle:
>>
>>    Length   Latency(us)    Bandwidth(MB/s)
>>         0       2.684          0.000
> 
> [...]
> 
> Nice work, Patrick - but such numbers are of little value if the
> benchmark used to get them is not stated. I'd recommend mpptest (from 
> MPICH). Plus, the compiler etc. is also of interest when it comes to 
> latencies.

Thanks. Such numbers have always little value coming from a vendor. My 
point was simply that >10us was not really today's ballpark.

For your curiosity, it was using an in-house MPI Pingpong (one message 
at a time, not a bogus pipelined pingpong used to confuse people and 
make big pipes look good). For very small messages, most of Pingpong 
codes are similar, compiler has no impact (it was using the gcc that was 
installed on the machine at that time). For asymptotic bandwidth, the 
major difference is the way you compute 1 MB, either 1024*1024 Bytes, or 
1000000 Bytes. In the networking world, it tends to be 1000000 Bytes.

Patrick
-- 

Patrick Geoffray
Myricom, Inc.
http://www.myri.com