[Beowulf] cisco networking

[Michael Di Domenico]

On Sun, Jan 10, 2010 at 1:29 AM, Glen Dosey <doseyg at r-networks.net> wrote:
> You can have multiple ether-channel links between 2 switches. The
> limitation is that if they are in the same layer 2 broadcast domain
> (VLAN) only 1 will be active and STP will block the other.
>
> If you place each switch in a separate network you could use equal cost
> load balancing across multiple point to point SVI's on ether-channel
> links. You'll increase bandwidth at the cost of a little additional
> latency. Of course at the bandwidths you are talking about (160 Gbit/s
> or more ) I have no idea how it would really work. Backplane bandwidth,
> hashing algorithms and all sorts of other factor come into play and
> could cause it to fail completely. I'd love to try it out but I can't
> help but think there is probably a better architectural solution.
>
> What are you trying to do ?

The attempt is really to get a bunch of devices which only have
ethernet 1Gbps connectivity, to talk to a bunch of other machines with
Infiniband Connectivity.  (Ie, compute nodes to storage).  But we want
the aggregate bandwidth to be over 5GBps.

I was trying to see if there was a simple solution using some of the
existing stuff we have, without having to get overly creative with the
network or purchase 10G Ethernet to IB routers...

Looks like IB routers is the only solution that really works for me...