[Beowulf] choosing a high-speed interconnect

[Joe Landman]

First questions first:

  Why do you think you need a faster network, and what aspect of fast do 
you think you need?  Low latency?  High bandwidth?

Then...

  What codes are you running?  Across how many CPUS?  Have you done a 
performance analysis on your system to observe "slow" runs in progress, 
and are you convinced that the network is the issue?

We have done lots of tuning bits for customers where the issues wound up 
being something else than what they had thought.  It is worth at least 
looking into for your code/problems, and identifying the bottleneck (if 
you haven't already done so).

That said, all the below require an external "switch" fabric.  All range 
from $500-$2000 per HBA, and about $1000 or more per switch port.  
Varies a bit.  Performance is comparible in most cases, with IB  seeming 
to have a higher ceiling than the others.

Joe

Chris Sideroff wrote:

>I'm sure posing this may raise more questions than answer but which
>high-speed interconnect would offer the best 'bang for the buck':
>
>1) myrinet
>2) quardics qsnet
>3) mellanox infiniband
>
>Currently, our 30 node dual Opteron (MSI K8D Master-FT boards) cluster
>uses Gig/E and are looking to upgrade to a faster network. 
>
>As well, what are the components would one need for each setup?  The
>reason I ask is for example the Myrinet switches accept different line
>cards and am not sure which one to use.  Sorry if this a bit of a newbie
>question but I have no experience with any of this kind of hardware. I
>am reading the docs for each but thought your feedback would be good.
>
>Thanks
>
>Chris Sideroff
>
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>  
>

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 612 4615