very high bandwidth, low latency manner?

[Craig Tierney]

On Fri, Apr 12, 2002 at 07:50:26PM +0200, Steffen Persvold wrote:
> On Fri, 12 Apr 2002, Craig Tierney wrote:
> 
> > On Fri, Apr 12, 2002 at 01:40:08AM -0400, Patrick Geoffray wrote:
> > > Steffen Persvold wrote:
> > >
> > > >>I talked to a guy at SC2002 from Quadrics and he said
> > > >>that list pricing on a Quadrics network was about $3500
> > > >>per node when you are in the 100s of nodes and up.
> > > >>The price includes the cards, cables, switches,
> > > >>etc.  This doesn't include any sort of discount that you
> > > >>might get.  Myrinet is about $2000 for an equivelent
> > > >>network at list price.   Dolphin/SCI falls around $2245 list
> > > >>per node (if the system is > 144 nodes and you have to get
> > > >>the 3d card).
> > >  >
> > > > This is list prices for the cards only, right ?
> > >
> > > Not for Myrinet. Actually $2000 per node is the total cost
> > > (NIC/cable/port/software) for the high-end products (with L9/200 MHz),
> > > should be more like $1500 for low-end ones. Craig is spoiled, only buys
> > > the top stuff :-)
> >
> > Sorry Patrick.  The problem with trying to state numbers is that
> > if you get it wrong, then the real knowledgeable ones can point it out.
> >
> > I figure out list cost on a 256 node system at about $2000 before for
> > basic hardware.  I as wrong.  I reworked it and it is $1500 for
> > 256 (and would be the same for 512 and 1024).

Your calcuations are fine.  I shouldn't be allowed to add and
multiply numbers.  When I redid the numbers I redid them 
incorrectly.  From list prices, cables are about $100 each,
and you need two per card.  So add about $200 to your prices.

> 
> So what is wron with my calculations :
> 
> 256 node L9/2MB/133MHz config :
> 
> M3F-PCI64B-2 NICs           256 *  $1,195 = $305,920
> M3-E128 Switch enclosures     6 * $12,800 =  $76,800
> M3-SW16-8F "Leaf" cards      32 *  $2,400 =  $76,800
> M3-SPINE-8F "Spine" cards    64 *  $1,600 = $102,400
> -----------------------------------------------------
> Total cost                                = $561,920
> Node cost                                 =   $2,195
> 
> and for a L9/2MB/200MHz config :
> 
> M3F-PCI64B-2 NICs           256 *  $1,495 = $382,720
> M3-E128 Switch enclosures     6 * $12,800 =  $76,800
> M3-SW16-8F "Leaf" cards      32 *  $2,400 =  $76,800
> M3-SPINE-8F "Spine" cards    64 *  $1,600 = $102,400
> -----------------------------------------------------
> Total cost                                = $638,720
> Node cost                                 =   $2,495
> 
> And this is without cable cost (since I don't quite know the cable
> requirements for the total system, but atleast it is approx $100 per
> node).
> 
> >
> > I thought I had decent information.  What I was trying to provide was
> > info on the three options.  The SCI price is per node (cards, cables,
> > software).  It is $2245 list.  This would be appropriate for systems over
> > 144 nodes.
> >
> 
> Actually, Wulfkit3 comes in two flavors wether you want 1U or can manage
> with a 2U (or higher) solutuion. The 1U is $2,445 and the 2U solution is
> $2,245 per node.
> 
> > The Quadrics number I got from the rep might have been the sales number,
> > and not a perfect comparsion to all the required hardware for a system of
> > a few hundred nodes.
> >
> 
> Now we have price comparisons for the interconnects (SCI,Myrinet and
> Quadrics). What about performance ? Does anyone have NAS/PMB numbers for
> ~144 node Myrinet/Quadrics clusters (I can provide some numbers from a 132
> node Athlon 760MP based SCI cluster, and I guess also a 81 node PIII ServerWorks
> HE-SL based cluster).

I don't think that anyone is going to have numbers on the same hardware.
Too bad.  It would be interesting to see the differences.  However,
that may end all the discussions and that would be no fun.

Craig