[Beowulf] Gbe switch comparision

[Felix Rauch]

On Thu, 30 Sep 2004, Håkon Bugge wrote:
> we did a study of two different Gigabit ethernet switches from one
> particular vendor on a 16-node XEON cluster (2.4GHz dual XEON,
> 400MHz FSB, Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet
> (rev 15)). We run a number of benchmarks every night on the cluster
> as part of our regression testing for Scali MPI Connect(synthetic,
> application benchmarks and applications).  Although we ran on a
> _very_ little system, that is not of the fastest these days, we
> observed significant differences. I thought the findings might be of
> interest to the community, not necessarily as a guide to which
> switch to purchase, but merely as a motivation to assess the quality
> of the switches to consider in a purchase.

We found similar differences when comparing two Fast Ethernet switches
for the 128-node cluster "Xibalba" at ETH Zurich. The cheaper of the
two swiches was advertised with full performance, which it simply
couldn't fulfill. Our comparison was done with microbenchmarks as well
as different application benchmarks (including HPL). We also compared
these numbers with a very cheap and limited maintenance network (based
on a tree of small switches). While the cheap maintenance network had
a much lower performance, its price/performance ratio was actually
much better than for the other switches.

If you are interested in more details, have a look at the following paper:

Christian Kurmann, Felix Rauch, Thomas M. Stricker: "Cost/Performance
Tradeoffs in Network Interconnects for Clusters of Commodity PCs",
Proceedings of Workshop on Communication Architecture for Clusters,
April 22, 2003, Nice, France.

The full paper is here:
http://www.cs.inf.ethz.ch/stricker/cops/cac03/cac03-kurmann-rauch-stricker.pdf

The slides of the talk (with some additional graphcs):
http://www.cs.inf.ethz.ch/stricker/cops/cac03/cac03-talk-rauch.pdf

- Felix