[Beowulf] Re: Re: Home beowulf - NIC latencies

[Maurice Hilarius]

----------------------------------------------------------------------

>Message: 1
>Date: Thu, 10 Feb 2005 10:15:54 +0100
>From: Joachim Worringen <joachim at ccrl-nece.de>
>Subject: Re: [Beowulf] Home beowulf - NIC latencies
>
>Patrick Geoffray wrote:
>  
>
>>Seriously, here are MPI latencies with MX on F cards on Opteron (PCI-X), 
>>that includes fibers and a switch in the middle:
>>
>>   Length   Latency(us)    Bandwidth(MB/s)
>>        0       2.684          0.000
>>    
>>
>[...]
>
>Nice work, Patrick - but such numbers are of little value if the 
>benchmark used to get them is not stated. I'd recommend mpptest (from 
>MPICH). Plus, the compiler etc. is also of interest when it comes to 
>latencies.
>
>   Joachim
>
>  
>
True, but it does not change the facts.
Further, all of these lovely benchmarks lack one really important detail:
Comparisons between different interfaces and drivers MUST show CPU usage 
while running them.
If I have a fantastic device that uses infinitely small time (latency) 
and moves huge amounts of data (bandwidth) but in doing so it takes 80% 
of a CPU, we do not have a useful solution..
That is where Myrinet and Quadrics shine, and also this is the detail 
that the various OB vendors carefully dance around.
All the communications performance in the world does not matter if it 
consumes a large amount of CPU cycles.

A further test that some vendors artfully avoid is the actual latency of 
all nodes in a cluster across the switching device.
I have seen a number of "benchmarks" showing great numbers, but on 
looking closer a great number of them are either on two computers, 
directly connected, or are on switching networks that use a number of 
small switches, and they do not show the worst case latency across all 
the switches, on the greater number of hops.

So, your points are excellent, Joachim,  but I have to say that even 
greater degrees of information are needed before any meaningful 
conclusions may be drawn.

What we all need is some form of useful standardized benchmarks that 
looks like real world code from a number of different disciplines, that 
we can use to test the hardware, so we may compare results in a 
meaningful manner.


With our best regards,

Maurice W. Hilarius        Telephone: 01-780-456-9771
Hard Data Ltd.  FAX:       01-780-456-9772
11060 - 166 Avenue         email:maurice at harddata.com
Edmonton, AB, Canada       http://www.harddata.com/
   T5X 1Y3

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20050210/feb53a3e/attachment.html>