[Beowulf] cisco networking

[Mike Davis]

Greg Keller wrote:
>
>>
>> Date: Mon, 11 Jan 2010 09:58:24 -0500
>> From: Michael Di Domenico <mdidomenico4 at gmail.com 
>> <mailto:mdidomenico4 at gmail.com>>
>> Subject: Re: [Beowulf] cisco networking
>> To: Glen Dosey <doseyg at r-networks.net <mailto:doseyg at r-networks.net>>
>> Cc: Beowulf at beowulf.org <mailto:Beowulf at beowulf.org>
>> Message-ID:
>> <e75d22a91001110658k6b5e30eaq83e8a83080cc529 at mail.gmail.com 
>> <mailto:e75d22a91001110658k6b5e30eaq83e8a83080cc529 at mail.gmail.com>>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> On Sun, Jan 10, 2010 at 1:29 AM, Glen Dosey <doseyg at r-networks.net 
>> <mailto:doseyg at r-networks.net>> wrote:
>>> You can have multiple ether-channel links between 2 switches. The
>>> limitation is that if they are in the same layer 2 broadcast domain
>>> (VLAN) only 1 will be active and STP will block the other.
>>>
>>> If you place each switch in a separate network you could use equal cost
>>> load balancing across multiple point to point SVI's on ether-channel
>>> links. You'll increase bandwidth at the cost of a little additional
>>> latency. Of course at the bandwidths you are talking about (160 Gbit/s
>>> or more ) I have no idea how it would really work. Backplane bandwidth,
>>> hashing algorithms and all sorts of other factor come into play and
>>> could cause it to fail completely. I'd love to try it out but I can't
>>> help but think there is probably a better architectural solution.
>>>
>>> What are you trying to do ?
>>
>> The attempt is really to get a bunch of devices which only have
>> ethernet 1Gbps connectivity, to talk to a bunch of other machines with
>> Infiniband Connectivity.  (Ie, compute nodes to storage).  But we want
>> the aggregate bandwidth to be over 5GBps.
>>
>> I was trying to see if there was a simple solution using some of the
>> existing stuff we have, without having to get overly creative with the
>> network or purchase 10G Ethernet to IB routers...
>>
>> Looks like IB routers is the only solution that really works for me...
>>
>>
>> -
Cisco switched can be connected via trunked links. This allows you to 
use multiple links to increase the bandwidth. The limit used to be 3 but 
that may have changed. You can check the switche's online documentation 
for more information.