[Beowulf] A Cluster of Motherboard.

[Andrew Piskorski]

On Fri, Nov 11, 2005 at 10:04:40AM -0500, Douglas Eadline wrote:
> I could no agree more - use cases.

Tower cases are large, bulky, not all THAT cheap, and (depending
somewhat on the case) often fairly awkward to work on.

After some initial experimentation, my tentative favorite chassis
setup for a bare-board cluster is...  cookie trays!

The very sturdy 18" x 26" 16 gauge aluminum cookie trays (sheet pans)
are $9 each, slightly less for the still sturdy 18 gauge version.
More importantly, pre-made racks ("tray trucks") are readily available
to hold all those trays.  Depending on shipping and whether you want
the 19 or 39 slot model, the tray truck costs around $160 to $240 or
so, including shipping.  acitydiscount.com had the best prices, the 39
slot model for $208 w/ free shipping, but they no longer have them
listed at all on their website, I'm not sure why.

  http://www.bigtray.com/productdetails.asp!sku.ADV18A26,catid.14810.html
  http://www.bigtray.com/productdetails.asp!sku.ADV188A26,catid.14810.html

  http://www.nexelshelf.com/traycart.htm
  http://www.globalindustrial.com/gcs/product/productsPerPicGroups.web?picGroupKey=2478&options.parentCategoryKey=165
  http://www.anytimeproducts.com/meg/wiretraycarttruck.htm

  http://nexel.superiorshelving.com/cgi-bin/nexel/shop.cgi?c=detail.htm&itemid=TT39CEL
  http://nexel.superiorshelving.com/cgi-bin/nexel/shop.cgi?c=detail.htm&itemid=TT39C
  http://nexel.superiorshelving.com/cgi-bin/nexel/shop.cgi?c=detail.htm&itemid=TT19CEL
  http://nexel.superiorshelving.com/cgi-bin/nexel/shop.cgi?c=detail.htm&itemid=TT19C

  http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&refid=store&item=7553415980
  http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&refid=store&item=7553415656
  http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&refid=store&item=7558871793
  http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&refid=store&item=7558785768

You can easily fit two micro-ATX motherboards in multiple orientations
onto each tray, with room left over for power supplies, disk drives,
or etc.  Two ATX boards will also fit as long as you orient their rear
output panels facing away from each towards the two short ends of the
tray.

Unfortunately, although two of the larger-than-ATX boards often used
for servers would just barely fit ON the tray, the overlap of the tray
truck mounting rails means that for most motherboards it would be
probably be impossible to then slide the tray into the tray truck.
(Darn!)  You could of course easily use only 1 such board per tray if
you wanted to, e.g., for motherboard testing purposes.

The Nexel tray trucks have slots 1.5" apart, which means that with no
PCI cards at all, you can easily fit 1 tray every 2 slots.  With
REALLY short PCI cards, you should still be able to fit trays every 2
slots.  E.g, an Intel Pro/1000 MT gigabit adapter will NOT fit with
even the compact mounting bracket, but if you remove the bracket, it
fits.  With trays every 3 slots, all compact PCI cards will fit
easily, including their short brackets.  A true full-height PCI card
probably still won't QUITE fit, even with it's bracket removed.

With the 39 slot truck and 19 trays, that means you can fit c. 38
nodes per each (32x18x69") "rack".  Cost for the rack and trays is
around $380, c. (9*19 +208*1)/(19*2) = $10 per node.  If go for the
roomier 3-slot tray spacing, that's 13 trays and 26 nodes per rack,
about $12.50 per node.

That's nominally cheaper than even the cheapest tower cases, while
being more compact, and friendler for a development/hobbyist
environment where you want to pull out a node and work on it, etc.

For comparison, a smallish tower case is about 8x19x17".  38 of those
are 57 ft^3.  With small mini-tower cases, you might get that down to
40 or so.  The tray truck above is 23 ft^3.

The aluminum trays drill easily, and don't seem to leave too many
hanging little burrs to fall and short something out, although they do
leave SOME, which you'll want to clean off carefully.  Which is yet
another reason I find Jim Lux's idea for attaching motherboards with
double-sided foam tape (no drilling!) very interesting...

The trays are constructed with a thick steel bar running inside the
rolled over lip of the tray.  If you really really want to butt the
rear connectors of a motherboard flush to the edge of the tray without
blocking the connectors, you can use bolt cutters to cut the steel rod
in two places, then grab the vertical aluminum lip with a pair of
pliers, and roll it down.  Rolling it down isn't hard, and gives nicer
results than cutting it off.  (Yes, I tried it, on an 18 gauge tray.)
None of that is necessary though.

> http://www.clustermonkey.net//content/view/41/29/
> 
> As I mentioned in the article, the best "find" was a Super-Flower Micro-ATX
> case for $40. It was small, well designed and did not have sharp edges.
> Unfortunately it has been discontinued, but I am sure you can find a

My recent favorite was the Skyhawk/Eagle BAX-4618 case.  It also is no
longer available, but a few months ago Newegg had a special on them
for $10 each, which I think was about $25 with shipping.

The BAX-4618 approaches elegance in its utter cheapness, as unlike
nearly all cheap cases, it seems to have been fairly thoughtfully
designed.  It has 120 mm fan mounts both front and back, the front fan
blows over the hard drives, the points for screwing down PCI cards are
unusual but work quite well, etc.  It also looks pretty good.

The case's only real flaws are that it does have a few sharp edges
(not that many), the front 120mm fan bracket fits only a 1" rather
than 1.5" thick fan, and the airflow from the front 120 mm fan is
somewhat obstructed by all the stupid plastic gimickry on the front of
the case.

AFAIK NewEgg is the best place to shop for tower or desktop cases.
Their killer feature is that unlike everyone else, they actually
provide enough PICTURES of every single case, both inside and out, to
give you a decent idea of what you're actually getting, whether it's
cooling is any good, etc.

  http://www.newegg.com/ProductSort/Category.asp?Category=9

Unfortunately, decent SMALL cheap cases with good airflow don't seem
to exist.  Anything smaller than mid-tower ATX is uncommon to start
with, and hitting any 2 of the three "small, cheap, good cooling"
variables seems to be hard, never mind all 3.

If you don't mind the extra noise (e.g., for a cluser), it might be
nice to be able to stick 1U server power supplies (inexpensive, high
quality, active PFC, etc.) into a consumer tower case, but such cases
never come with with mounting brackets or holes for that.  (And if
you're willing to drill holes and such, you probably wouldn't be
choosing tower cases for a cluster anyway.)

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/