[Beowulf] 512 nodes Myrinet cluster Challanges

[Bill Broadley]

> in the cluster above, I choose a chassis (AIC) which has a large centrifugal

Which?  I noticed some of their designs redirect all heat from the powersupply
into the side of the rack.

> blower, rather than a bunch of 40mm axial/muffin fans.  a much larger cluster
> I'm working on now (768 nodes) has 14 40mm muffin fans in each node!  while
> I know I can rely on the vendor (HP) to replace failures promptly and without
> complaint, there's an interesting side-effect: power dissipation.  of 12 fans
> pointing at the CPUs are actually paired inline, and each pair is rated to 
> dissipate up to 20W.  so a node that idles at 210W and 265W under full load

Wow, 10 watts per 40mm fan?  Really?

I've a broshure from Sunon handy:
40x40x28mm  9200rpm 2.6 watts
40x40x28mm 11000rpm 4.0 watts
40x40x28mm 13000rpm 6.8 watts.

The 56mm is 15.5 watts, but having 12 fans that large seems bordering
on insane.. I'd expect there wouldn't be much room for anything else.

That would be a square of fans, er 16cm on a side

> can easily consume 340W if the fans are ramped up.  ouch!

75 watts difference, hrm, 75/12 = 6.25 watts difference per fan between
the low rpm cool state and the hot state...  larger than I'd guess.

Maybe HP uses substantially more power hungry fans than sunon.
 
-- 
Bill Broadley
Computational Science and Engineering
UC Davis