[Beowulf] A Cluster of Motherboard.

[Glen Gardner]

I think the advantage to stacking bare boards is the potential for
greater density in terms of number of cpus, and a potential for better
thermal management.  There is nothing to stop one from forcing massive
amounts of air through closely spaced boards to get the density up so
that large clusters take up a lot less valuable floor space.

The cost of an aluminum plate and standoffs is trivial compared to a
case, and once you have made one plate all the others can be readily
mass produced using a template and a hand drill.

I can fabricate a chassis for a mini itx cluster from aluminum plate and
aluminum angle stock with simple hand tools in about 2 days.... and I
could do it in hours if I owned a drill press and cutoff saw.

The main point is, it can be done cheaply and with simple tools.

There is nothing unprofessional about such setups... they are very
professional and offer layout flexibility that opens up new
possibilities.

My present setup has vertically arranged mini itx boards bolted to a
rack panel.  One could easily stack 18 mini itx form factor p4
motherboards with power supplies and hard drives behind a 12 inch high
rack panel, and have 7 of those arrays in a single equipment rack for a
total of 126 nodes in a single equipment rack.  with the vertical P4
boards you would have to force cooling air through the rack, but it will
work fine.

If you use dual cpu mini itx boards, that gives you 252 cpu's and a much
easier cooling solution.... I am sure that one can stack them in an even
denser fashion if they tried....

It is all about cpu density vs floor/rack space and thermal management,
and I am surprised that someone has not come up with a low cost
commercially available cabinet for beowulf clusters that embodied this
idea...


Glen Gardner




On Thu, 2005-11-10 at 10:27 -0800, Jim Lux wrote:
> At 08:54 AM 11/10/2005, Andrew Piskorski wrote:
> > On Thu, Nov 10, 2005 at 05:35:03AM -0800, Jim Lux wrote:
> > 
> > > However, lots of people have successfully built clusters from
> > stacks of 
> > > mobos.  I think the biggest one (in terms of # of nodes) is the
> > one with a 
> > > dozen or so Via mini-ATX boards.  I don't know that I've seen any
> > bare 
> > > bones clusters with more than 20 nodes.
> > 
> > Oh, I know of at least 3 bare-board clusters much larger than that
> > (and some smaller):
> 
> Excellent examples.. (what comes from making an off the cuff comment
> before having my first cup of coffee in the morning)..
> 
> >   http://jessen.ch/ammonite/
> 
> Telling comment:
> 
> "Perhaps the most helpful thing I could say is to urge you to consider
> building a conventional cluster (shelves of COTS midtower cases or
> racks of 1U pizza boxes) instead of something like ammonite. The
> ammonite design has some advantages (high cpu density, better
> ventilation and lower delta-T, for example), but designing and
> building it was a colossal time sink. ... 
> There were many little things that had to be custom made or modified,
> no one of which was a big deal, but all of which together were a very
> big deal. " (ellipses mine, JL)
> 
> 
> 
> >   http://joule.bu.edu/~hazen/LinuxCluster/
> 
> 
> "Instead of a flat plate we use a custom-made aluminum box with
> punched holes and welded corners made by a local sheet-metal
> house. ..."
> Dividing out their $4500 hardware cost, they spent just under
> $100/mobo for packaging (their box was probably about half that).
> Note well, they don't mention labor costs.. If you go custom box, you
> want to look closely at boxes designed for mass production: sliding
> Tab A into Slot B is a heck of a lot faster than fumbling for 6-32
> screws.
> 
> >   http://krone.physik.unizh.ch/~stadel/zBox/
> 
> "
> We greatly acknowledge the aid of the Physics Mechanical Workshop at
> the University of Zurich for: 1) turning the "napkin-sketch" into a
> proper CAD/CAM design of the machine; 2) providing numerous
> suggestions which improved the detailed design; 3) providing a
> gigantic room for the construction of the boards; 4) and, well,
> building the thing! 
> "
> 
> ---
> In any case, ALL of these bigger systems had fairly custom designed
> (read: not cheap) packaging hardware.  They are pretty nifty looking. 
> 
> 
> They also raise some "serviceability" questions too...
>  
> Jim 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf