large clusters and topologies

[Patrick GEOFFRAY]

Steffen Persvold wrote:

> > In reality, the biggest SCI cluster that i know is in Paderborn :
> > 96 nodes. I have talked in Mannheim with people who worked on it,
> > and it seems to be working as 3 clusters of 32 nodes.
> 
> Not true, they have one 32 node system and one 96 node system. I was
> personally there to set it up, so I ran a Bisection bandwidth test on
> all 96 nodes with MPI (numbers are from mpptest from the MPICH suite):

Sorry, I heard that from the SuperComputer event in Mannheim, one
guy who worked on the machine. I use "seems" as i cannot verify
this information.

> The total aggregated application network bandwidth for 96 nodes was
> measured to:
> Bandwidth:  3124.29 MByte/s at size 524288  bytes(peak)

That's 32 MB/s per node, 64% of the PCI bus assuming a PCI 32/33
at 100 MB/s of useable bandwidth.

> BTW: Aerospatiale also has a system (4 nodes)

Aerospatiale is a HUGE company and it's hard to know the others
projects :-) (I have seen people doing the same thing at the same
time).
We have a SCI cluster with 8 nodes for evaluation. But the cards
don't provide the mezzanine extension. I was quite disapointed by
the aggregate bandwidth, the scalability and the stability. We are
interested by large bandwidth for multimedia applications, so we
have chosen Myrinet, but SCI may fit the requirements of other
applications.

Thanks for the input.

Patrick Geoffray
---
Aerospatiale Matra - Sycomore
Universite Lyon I - RESAM
http://lhpca.univ-lyon1.fr