[Beowulf] Infiniband modular switches
patrick at myri.com
Mon Jul 14 10:42:07 PDT 2008
Gilad Shainer wrote:
>> The injection rate is irrelevant
> The injection rate is super relevant. If your injection rate is 10% of
The injection rate is absolutely irrelevant for contention due to
Head-of-Line Blocking. You will have the same fabric efficiency under
contention if your link rate is 1 Mb/s or 100 Gb/s. For example, the
efficiency of a single-channel *full* crossbar under random traffic is
58.6%, whatever is the link rate.
Google "HOL blocking" or read a book.
> you referred to, when doing the same testing on IB DDR we got much
> better results with IB versus Quadrics. your theory does not really meet
The routing efficiency of 3-year old Quadrics QsNetII is greater than
latest Mellanox IB in these tests. For IB SDR link rate, Quadrics has
more throughput because of the better efficiency, as their link rate is
similar. For DDR throughput, it is possible that the increased link rate
compensate for the bad routing efficiency.
The only comparison I have seen with IB DDR was by Woven Systems at
Sandia last year, and 10 Gb/s Ethernet (higher link rate than QsNetII)
was beating IB DDR for a similar benchmark.
> reality. Also, when you do look on adaptive routing, make sure it is
> real time solution... Not some synthetic testing ....
AlltoAll of large messages is not a useless synthetic benchmark IMHO.
> What Panda did is using multiple static routs, which can be chosen
> according to different algorithms. It is an interesting idea, and used
> in several cases with nice results. This is not adaptive routing.
That's right, this is not adaptive routing, it's lip stick. And I am
sorry but those are not nice results compared to real adaptive routing.
More information about the Beowulf