Xbox clusters?

[Robert G. Brown]

On Wed, 28 Nov 2001, Velocet wrote:

> On Wed, Nov 28, 2001 at 02:04:55PM -0500, Velocet's all...
> > On Wed, Nov 28, 2001 at 12:50:46PM -0500, Josip Loncaric's all...
> > > Microsoft's Xbox packages a 733 MHz Pentium III, 64 megabytes of memory,
> > > a DVD drive, 100 Mbps Ethernet, and an 8-gigabyte hard disk for about
> > > $300.  This would make it a reasonably powerful cluster node with an
> > > excellent price/performance ratio.  Of course, the thing runs a
> > > slimmed-down variant of Windows 2000 instead of Linux, but has anyone
> > > discussed making an Xbox cluster?
> > 
> > Why bother when for about $300 USD you can put together a
> > cluster node with a 1.333GHz athlon with 256Mb of DDR ram?
> > 
> > Sides, who brought 'price/performance' onto this list? Dont know thats never a
> > factor on the beowulf list? :)
> 
> So, the question is, with these numbers, how do people end up spending
> $250K on 40 or even 60-CPU clusters?

Well, start with $300 rackmount cases (a rackmount case alone can easily
cost more than an Xbox).  Add a high end P4 motherboard, the fastest
P4-Xeon, and fully populate the MoBo with the biggest, most expensive
RDRAM sticks you can find.  Get a big, fast SCSI drive and controller.
Finish off with the fastest network you can arrange.

The high speed network alone can cost $2K/node, and one can easily
enough spend $2K on a rackmount P4 node (exclusive of the high-speed
network).

Besides, a lot of the top-end numbers are (or at any rate were)
generated by alpha/myrinet clusters, where individual nodes could easily
run $6K, with discount, NOT including the network, maybe $8K/node
including the network.  One could drop more than $500K on a 64 node
cluster without even breaking a sweat.

Note that this sort of high end cluster was (and really still is)
appropriate for moderately fine-grained parallel computations, where one
needs to spend proportionally much more for the network than usual, and
where the fastest possible processors with the fastest and biggest
memory can help control the ratio of serial code fraction to parallel
code fraction, allowing one to actually scale an application UP to 64
nodes.  

Yes, one might be able to afford 1000 AMD nodes on some agglomeration of
daisy chained switches for the same $500K (if you could afford to house
and feed them given that they would consume some 100 KW or more in
operation).  Yes, those 1000 nodes might have 2-3x the aggregate power
of the really expensive cluster for the same money.  However, if
>>your<< problem only scales to 6 nodes with that ratio of CPU speed to
network speed, the giant AMD cluster is obviously not smart.

There is a tremendous range of variation in cluster designs, with all
sorts of mixes of investment in node speed, memory speed, network
topology and speed, and while the "standard recipe" beowulfish cluster
(pile of PC's, switched 100BT, linux) is right for some (indeed, right
for me:-) it isn't right for everybody.

So Josip's question was really relevant and one that we've kicked around
on this list some before -- one day game systems may well be viable
candidates as nodes.  I don't think the Xbox is there yet.  The
new/future Sonies may be, but I'm not so certain.

The problem is:  All PC's can play games, many of them as well or better
than a dedicated gaming box.  PC's can do much more -- they are general
purpose.  The parts for a PC are all commodity and largely
interchangeable.  These factors conspire to keep PC's as powerful AND
cheap as they can reasonably be.  Game boxes nowadays have to be able to
do nearly everything a PC can do -- a motherboard with integrated
graphics, sound and network is just about a game box on a board, lacking
only an operating system and some I/O channels.  There is such a small
and narrowing window in between these two extremes that I'm not at all
convinced that there will EVER be an advantage in using game systems as
nodes.  By the time they have the features and expandability of a
PC-based node, they will necessarily reach the PC in price point or
somebody will just repackage the node and sell it as a PC (and so reach
the PC in price point).

Anyway, over the many years I've seen "thin" or "special purpose"
systems of all sorts come with much hooraw and seen them go again like
thieves in the night, with most souls sorry they ever bought them.  The
general purpose cost/benefit sweet spot is right in the middle of the PC
commodity market because market forces evolve it that way, and only
rarely does a processor based "computational" design (excluding the vast
world of controllers) come along that really can sustain a special
purpose market let alone be backportable to general purpose use.

This is the Lesson of the Wang.

(At least for those of you old enough to remember what one is...:-)

    rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu