[Beowulf] Infiniband modular switches

[Ashley Pittman]

On Tue, 2008-08-26 at 12:50 -0700, Gilad Shainer wrote:
> Dan Kidger wrote:
>  
> > Gilad wrote:
> > > It was proven by the same person who did the slides you 
> > referred to, 
> > > when doing the same testing on IB DDR we got much better 
> > results with 
> > > IB versus Quadrics. your theory does not really meet reality.
> > 
> > Care to describe to the list what these results were, and to 
> > speculate about why IB gave better results?
> > 
> > Daniel
> > 
> Sure. My intend was not to compare between IB and Quadrics, but to show
> that the network/host injection rate is very relevant in contention
> tests. For given host injection rate, increase in network capability
> reduces the contention (assuming finite number of hosts.....). As for
> the numbers:
> 
> Send ave: IB SDR 263MB, Quadrics 369MB, IB DDR 453MB
> Receive ave: IB SDR 340MB, Quadrics 370MB, IB DDR 547MB
> 
> I noticed that Quadrics decided to use just part of those numbers for
> their poster at ISC ...  ;-)

I don't remember the DDR figures being on the original source of the
data, were the DDR figures measured using the same code?

Those figures have always annoyed me anyway as they test at a very small
message size so just represent one point on a very steep curve.  It says
little about the available bandwidth or any contention issues, the only
two things you can draw from it are the steepness of the graph (the mean
value) and the quality of the implementation (the range of values) and
on both measures Quadrics performed significantly better than IB.

It would have been nice to have re-done the test using a range of
message sizes which would have shown Quadrics showing a much higher
bandwidth figure, this would have completely defeated the point of the
benchmark in the first place however which was to show that adaptive
routing is necessary for consistent network performance.

Ashley.