large clusters and topologies

Joel Jaeggli joelja at
Fri Jul 28 12:01:32 PDT 2000

On Fri, 28 Jul 2000, Greg Lindahl wrote:

> > that and you can now get fast-ether switches with port-counts up to about
> > 384 ports which isn't to shabby compared to the denisty of most other
> > lan-interconnects.
> Actually, you've been able to get those for quite a while. Centurion at UVa
> was built in late 1998 with a 288 port switch, which was composed of 12
> 24-port switches with gigabit uplinks, and a 12-port gigabit switch. It's
> dense, but the particular stuff I usd (3com) was expensive in today's terms.
> Today you can both buy chassis switches with a gazillion ports (but often
> limited backplane bandwiths), or cobble together lots of cheaper switches
> such as HP 4000M procurves with copper gigabit uplinks and a gigabit switch.
> The same philosophy, but the details as to who makes the most cost-effective
> components keeps changing over time.

The one(high density switch) we have evaluated for use on our campus
backbone (rather than a cluster) was a foundry big-iron 8(were actually
deploy cisco cat6509's but thats a different story). while that was mostly
populated with gig interfaces, with fasts it would be 384ports with a
128Gb/s backplane. since 10 ports would be 2Gb/s (fdx) 100 would be 20 and
~400 would 80Gb/s or so which ought to be well within the margins of the
switch fabric, of course that doesn't much matter if it blocks on one port
(but if already decided to use ethernet that's not really an issue) but if
you need more ports than that you can interconnect switches using 10Gb/s
wdmi interfaces although in practice we're not using anything faster than
oc48 on campus.

given the foundry 22% edu discount they fall in line with commidity per
port costs.
> We tried to keep an up-to-date document describing how to build cheap
> ethernet switches for various sizes, but it was too hard to keep it from
> bitrotting.
> -- greg
> p.s. and, incredibly enough, if you want an 80 port HP4000M, it's still
> cheaper to buy 2 40-port boxes and throw away the second chassis than it is
> to buy expansion cards.

Joel Jaeggli				       joelja at    
Academic User Services			     consult at
     PGP Key Fingerprint: 1DE9 8FCA 51FB 4195 B42A 9C32 A30D 121E
It is clear that the arm of criticism cannot replace the criticism of
arms.  Karl Marx -- Introduction to the critique of Hegel's Philosophy of
the right, 1843.

More information about the Beowulf mailing list