[Beowulf] Multirail Clusters: need comments

[Greg Lindahl]

On Tue, Nov 29, 2005 at 09:38:15PM +0800, Greg Anderson wrote:

> I would like to know the views of the cluster gurus on multi-rail clusters,
> esp, quad rail networks. Are quad rail networks practical to implement ( say
> InfiniBand, Quadrics, Myrinet,  (GE, maybe not?)).

The one thing you can be sure of is that different interconnects have
different issues dealing with multi-rail. Even if the microbenchmarks
show an improvement, real apps often do not.

> - Are there any quad rail HPC clusters? (me and google couldn't find any)

Linux Networx has announced a 4-socket/2-rail and 8-socket/4-rail
InfiniPath system, the LS/X. Our experience on the 2-rail system is
that it scales quite well. We don't have that much experience with the
4-rail systems yet. Our approach to multi-rail is bind cpus to a
particular adaptor chip.

-- greg