[Beowulf] A Cluster of Motherboard.
James.P.Lux at jpl.nasa.gov
Fri Nov 11 07:36:16 PST 2005
At 06:58 AM 11/11/2005, Robert G. Brown wrote:
>On Fri, 11 Nov 2005, Jim Lux wrote:
>>If you have an unusual packaging concern.. maybe it needs to be in a
>>sealed box? maybe it has to fit into that antique grandfather clock?,
>>etc. then you've got a case for custom design.
>Or if the commercial packaging available is still being sold at a hefty
>premium, opening up a potential window of CBA advantage.
>Let us recall that all of these arguments might have been applied to the
>creation of beowulfs in the first place, when perfectly good big iron
>supercomputers that were de facto clusters were available. After all,
>it took lots of human time to set up that stack of PC's, buy shelving,
>screw around with wiring them together, and then (from a certain amount
>of bitter experience) wrestle with Linux (especially e.g. the SMP
>kernels and device driver issues) to get the whole thing to "work".
You raise an excellent point. Of course, the early beowulfs were
essentially hobby clusters, in that the builder was willing to spend a fair
amount of time for the thrill of creation. When you're doing it for the
first time, spending valuable research grant money (or equally valuable
uncompensated "free time") for labor to do it might be justifiable, if for
no other reason than you are helping to bring the per unit cost for those
who follow down the "learning curve".
It's the classic capital poor, time rich scenario that has driven artists
in garretts, inventors in their garages, and scientists in wherever poor
scientists work (a shack out back, a la Mme Curie?) to be creative. You
have a burning desire to answer some question, you can't afford a Cray to
answer it, so you improvise, and thereby develop something useful for the ages.
But.. there's a limit to the scale of such things. Have there been any
100+node hobby clusters (i.e. ones where there wasn't funding specifically
provided to get the cluster built), except perhaps the StoneSouperComputer
at ORNL? (having shot my figurative mouth off about this before, I expect
no less than 3 emails with links to specific examples of people building
dreadnought class 1000 processor clusters in their garage in their spare time).
Home/self built packaging is, I'd venture to guess, a way to do "proof of
concept" on a limited budget. Or, it's a somewhat unique artistic medium.
>The reasons that it >>was<< economically efficient to build beowulfish
>(order of magnitude ten, not even unity). Opportunity cost labor is
>trickier, but in most cases one would argue that it increases the
>cost-efficiency of clusters still further, at least in certain ranges of
>cluster size and with certain degrees of OC expertise available.
Exactly.. I'd say the size of 4-16 (just to keep it powers of 2) nodes is
where the "sweet spot" is for proof of concept/hobby/self packaged. You're
still at a reasonable overall power consumption (e.g. you can plug it into
one circuit and not start a fire, nor cause your a/c bill to skyrocket).
And, it's going to be in the "fit under or next to a desk" sort of size.
I'd argue that for >100 nodes, the job is bigger than one person can do, so
the issue of finding other "volunteers" comes in.
>That appears to be less true as blades are moving more towards the mass
>market and some low rent "micro board on cheap plate blades" appear --
>pretty much what you'd make yourself but done professionally and at an
>efficient scale, sans bells, whistles, and hefty markup.
I think, and others have also pointed out, there might be a market for
someone who makes some sort of cheap and cheerful vertical racking
hardware, adapted from existing "card cage" technology. That is, the "box"
is available already from some other source, and someone makes a flat plate
upon which a standard mobo and peripherals are mounted that slides into the
My previous experience with getting moderate quantities of sheet metal
enclosures made makes me think that the basic per unit cost of such a thing
might be in the $15-20 range, for quantities in the tens or low
hundreds. A few hundred dollars for NRE setup, then the rest is machine
time and handling (the raw material is steel, and exceedingly cheap)
More information about the Beowulf