Card/switch performance query

[Robert G. Brown]

Dear All,

We are getting ready to design and fund an upgrade for our existing
Intel-based environment.  I need data/advice on two issues:

  a) I'm generally dissatisfied with the performance of the Netgear
10/100 card -- in most of my tests it peaks out (too) far below wire
speed.  Some years ago I gave up 3c905's in favor of tulips because the
tulips outperformed them, were cheaper, and the 3c9xx cards had endless
driver problems.  I'm sure the world has moved on since then.  SO my
question:

  What 10/100 cards would you (any or all of you) recommend for use in a
"standard design" beowulf today?

Please address price, driver/card stability with current kernels, and if
at all possible, quantitative performance.  I can provide scripts to
generate a "sweep" of tcp streaming throughput measurements for a range
of message sizes from 1 to enough bytes to encompass a standard ethernet
MTU using either netperf or (preferrably) the (new) bw_tcp tool in
lmbench.  I find that the full sweep provides invaluable information as
well as reveals potentially serious performance problems.  I'd also be
interested in latency measurements (lat_tcp from lmbench or
request/response measurements in netperf).  Since latency and throughput
definitely depends on switch type, please indicate what switch type you
are using and whether it is cut-through or store-and-forward.

I'm particularly interested in comparative performance/reliability/cost
of the current 3c905, eepro100 and tulip-ish cards but would love to
hear about any others that might have emerged as strong competitors as
well.

  b) We'll very likely be upgrading/replacing our current aged Cisco Cat
5000 switch at the same time.  I expect that we'll need to support as
many as 64 nodes, although to connect to the department network proper
we'll probably have to go through an uplink to another (similar?)
switch.  The old Cisco was very nice in its day but has somewhat broken
Nway and we'd definitely like to get a switch that will last and give
decent (100 base) performance for the next few years.  On this list and
in recent discussion, the HP ProCurve 4000M has been suggested several
times by fairly smart folks (with the suggestion made by Don Becker,
IIRC, that one just buy two and put the cards and power supply from the
second back into the first to get an 80 port chassis, rather than either
get the 80 ports all at once or try to upgrade).  Another more recent
suggestion (from Christopher Hogue) was to get a Foundry Networks
Fastiron II, which is also the OEM of the HP 9000 series.  I suspect
that this is out of our budget range but I'd still be interested in
price/reliability/performance reports.

I know this is a perennial discussion item, but I have actually searched
the archives already and although there is some anecdotal stuff there
there aren't many true price/reliability/performance reports, especially
with quantitative reporting of the numbers.

In addition to your answers getting into the archives as reports in and
of their own right, I'll try to take the time to post an executive
summary when the thread peters out.

   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu