<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
----------------------------------------------------------------------<br>
<blockquote cite="mid200502102000.j1AK0Eb7016772@bluewest.scyld.com"
type="cite">
<pre wrap="">
Message: 1
Date: Thu, 10 Feb 2005 10:15:54 +0100
From: Joachim Worringen <a class="moz-txt-link-rfc2396E" href="mailto:joachim@ccrl-nece.de"><joachim@ccrl-nece.de></a>
Subject: Re: [Beowulf] Home beowulf - NIC latencies
Patrick Geoffray wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Seriously, here are MPI latencies with MX on F cards on Opteron (PCI-X),
that includes fibers and a switch in the middle:
Length Latency(us) Bandwidth(MB/s)
0 2.684 0.000
</pre>
</blockquote>
<pre wrap=""><!---->[...]
Nice work, Patrick - but such numbers are of little value if the
benchmark used to get them is not stated. I'd recommend mpptest (from
MPICH). Plus, the compiler etc. is also of interest when it comes to
latencies.
Joachim
</pre>
</blockquote>
True, but it does not change the facts.<br>
Further, all of these lovely benchmarks lack one really important
detail:<br>
Comparisons between different interfaces and drivers MUST show CPU
usage while running them.<br>
If I have a fantastic device that uses infinitely small time (latency)
and moves huge amounts of data (bandwidth) but in doing so it takes 80%
of a CPU, we do not have a useful solution..<br>
That is where Myrinet and Quadrics shine, and also this is the detail
that the various OB vendors carefully dance around.<br>
All the communications performance in the world does not matter if it
consumes a large amount of CPU cycles.<br>
<br>
A further test that some vendors artfully avoid is the actual latency
of all nodes in a cluster across the switching device.<br>
I have seen a number of "benchmarks" showing great numbers, but on
looking closer a great number of them are either on two computers,
directly connected, or are on switching networks that use a number of
small switches, and they do not show the worst case latency across all
the switches, on the greater number of hops.<br>
<br>
So, your points are excellent, Joachim, but I have to say that even
greater degrees of information are needed before any meaningful
conclusions may be drawn.<br>
<br>
What we all need is some form of useful standardized benchmarks that
looks like real world code from a number of different disciplines, that
we can use to test the hardware, so we may compare results in a
meaningful manner.<br>
<br>
<br>
With our best regards,<br>
<br>
Maurice W. Hilarius Telephone: 01-780-456-9771<br>
Hard Data Ltd. FAX: 01-780-456-9772<br>
11060 - 166 Avenue <a class="moz-txt-link-abbreviated" href="mailto:email:maurice@harddata.com">email:maurice@harddata.com</a><br>
Edmonton, AB, Canada <a class="moz-txt-link-freetext" href="http://www.harddata.com/">http://www.harddata.com/</a><br>
T5X 1Y3<br>
<br>
</body>
</html>