[Beowulf] cisco networking

[Greg Keller]

>
> Date: Mon, 11 Jan 2010 09:58:24 -0500
> From: Michael Di Domenico <mdidomenico4 at gmail.com>
> Subject: Re: [Beowulf] cisco networking
> To: Glen Dosey <doseyg at r-networks.net>
> Cc: Beowulf at beowulf.org
> Message-ID:
> 	<e75d22a91001110658k6b5e30eaq83e8a83080cc529 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> On Sun, Jan 10, 2010 at 1:29 AM, Glen Dosey <doseyg at r-networks.net>  
> wrote:
>> You can have multiple ether-channel links between 2 switches. The
>> limitation is that if they are in the same layer 2 broadcast domain
>> (VLAN) only 1 will be active and STP will block the other.
>>
>> If you place each switch in a separate network you could use equal  
>> cost
>> load balancing across multiple point to point SVI's on ether-channel
>> links. You'll increase bandwidth at the cost of a little additional
>> latency. Of course at the bandwidths you are talking about (160  
>> Gbit/s
>> or more ) I have no idea how it would really work. Backplane  
>> bandwidth,
>> hashing algorithms and all sorts of other factor come into play and
>> could cause it to fail completely. I'd love to try it out but I can't
>> help but think there is probably a better architectural solution.
>>
>> What are you trying to do ?
>
> The attempt is really to get a bunch of devices which only have
> ethernet 1Gbps connectivity, to talk to a bunch of other machines with
> Infiniband Connectivity.  (Ie, compute nodes to storage).  But we want
> the aggregate bandwidth to be over 5GBps.
>
> I was trying to see if there was a simple solution using some of the
> existing stuff we have, without having to get overly creative with the
> network or purchase 10G Ethernet to IB routers...
>
> Looks like IB routers is the only solution that really works for me...
>
>
> ------------------------------
>
Remember that you can actually use nodes to route IP between GbE and  
IB, just don't expect them to run at IB wirespeed.  This can at least  
save you from paying a premium for proprietary hardware in order to  
test functionality.  Last I tried 3-4 Gbit was about as fast as IP  
over IB would get through the Interface presumably due to IP  
overhead... but that was long ago so if you try it I'd love to hear  
your results.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20100111/13c321c6/attachment.html>