Facilities Requirements for Beowulf Clusters?

[Robert G. Brown]

On Sat, 12 Aug 2000, Don Morton wrote:

> Howdy, for years I've been running a 9-node cluster of 100MHz Pentiums,
> and I've gotten away with stuffing them into a single room with no
> additional power or air conditioning considerations.
> 
> I'm in a position now where I'll be getting 16 to 32 nodes, high-end,
> along with a high-end server, and I'm kind of worrying that a typical
> room's power system and, particularly, the air conditioning, might
> not be adequate.
> 
> I realise I can calculate the power requirements and compare them
> to what's available in the room, but A/C is a little tougher.  Have
> any of you tried jamming 16 to 32 nodes in a small room that's typically 
> kept at around 22C (72F) with any success?  I hear about a number of
> medium-sized systems like the one I'm getting ready to acquire, 
> and I wonder if people are really going all out to spend extra
> bucks on the facilities?
> 
> Thanks,
> 
> Don Morton
> 

I do a bit of an analysis of the A/C requirements for beowulfs in the
beowulf draft book available in pdf form from www.phy.duke.edu/brahma.
Basically, the rule is that all the energy that goes into the room has
to come out.  Some will come out through the walls and ceiling, driven
by the temperature difference between inside the room (hot, presumably)
and outside.  A bit might come out via convection patterns if the room
isn't sealed.  All the rest must be actively removed.

This is a VERY REAL thing to worry about.  16 nodes produce as much heat
as a good sized space heater, 32 nodes will damn near heat a small
house.  My home office (a largish 14x14 room) is close to ten degrees
warmer than the rest of the house in spite of air conditioning and gets
up in the 80-90 range very fast if the A/C goes off for any reason.  I
have only four nodes -- I don't think I could go to eight without
installing a window unit A/C (or keeping the rest of the house VERY
cold or installing some sort of fan to mix the room's air with house
air faster than the standard A/C vents do).

Our equipment room in the physics department is unfortunately small and
has no windows, one (solid) door (kept locked) and typically has 12-16
systems running therein.  If the A/C fails but the power doesn't, the
temperature inside soars over 100F very quickly -- we've experienced
hardware failures twice because of this and for a while had a thermal
kill switch on the most important systems (back when they were very
expensive and unique compared to our current stack of COTS boxes).

If you are counting on non-driven diffusion processes to keep a 16+ node
beowulf cool in a small room with no thermostat or significant air
mixing in a larger 72F ambient space, I doubt that you will comfortably
succeed unless you install e.g. a door exhaust fan that constantly mixes
the air inside and outside and hence transfers the heat to the building
A/C (presuming that it has the surplus capacity to deal with it).  If
you plan to go to 32 nodes (presuming 3200 watts, give or take a bit) I
suspect you'll need a ton or so of dedicated A/C (a ton of A/C is
defined in the book) with its own thermostat and possibly with a coupled
kill switch for the room power.

   rgb

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu