[Beowulf] Q: Cooling units? Raised floors? General machine room stuff..

[Jim Lux]

----- Original Message -----
From: "Brian Dobbins" <brian.dobbins at yale.edu>
To: <beowulf at beowulf.org>
Sent: Thursday, July 01, 2004 1:54 PM
Subject: [Beowulf] Q: Cooling units? Raised floors? General machine room
stuff..


> Hi guys,
>
>   I've recently put together a plan for a machine room in one of our lab
> spaces, but I'm not very knowledgeable about cooling (or infrastructure
> in general!), and would love to hear some thoughts on what the experts
> here think.
>
>   At the moment, a high-end power estimate is approx. 14.5kW for the
> systems we'll be running in the near future, and using Dr. Rob Brown's
> figures from his (excellent) writeup on Beowulf Infrastructure
> (http://www.linux-mag.com/2003-06/infrastructure_01.html), I'm
> estimating about 4.2 tons of AC needed.

That's not a huge A/C (typical home central air is 3-5 tons).

>
>   So here's the first of many questions - given the fact that we expect
> to expand our resources in the future, do we go with a much larger AC
> now?  Or is it possible/useful to simply add a second one later?  And
> are there any particular AC vendors people tend to recommend?

The question of future expansion could probably be best answered by looking
at some catalogs and seeing if there is a "chunk size" or granularity that
would work out well.  Also, think about the installation and facility
modification cost (it's not just buying the second a/c, it's running the
pipes and wires, laying a concrete slab for the compressor outside, etc.).
Most HVAC contractors can answer these questions pretty quickly, after
having seen your site.

>
>   Secondly, given this is a rather small machine room (aiming for a max
> capacity of 5 42U racks), what advantage would be found in going with a
> raised floor?  And does anyone have any cost estimates on raised floors
> for rooms on the order of, say, 15' by 15'?

Kind of depends on how you're arranging the racks.  If you've got them all
side by side in the middle of the room (so the cold air comes in the front
and blows out the back of a "wall o'racks") the raised floor probably isn't
worth it.  You'll have plenty of room on both the front and back for cable
trays, etc.  Raised floors are handy if you have rows and rows of racks, or
ugly cabling issues (where you need access to all sides of the rack without
tripping over cables) or if you're using cold air in at the bottom hot air
out at the top type flows.  Old style mainframes were designed for raised
floors, so the cabling comes out the bottom of the rack, and the airflow
likewise.
>
>   Finally, what other suggestions do people have for equipment needed?
> On my list we've got the UPS, the AC, thermal sensors and killswitches,
> APC Masterswitch units, power lines, network lines, web/video camera and
> an alarm.  Anything else people find useful?

Coat rack to hang some jackets on.

When thinking about UPSes, etc.  consider partitioning your system so that
not everything dies together.  Your monitoring and head node computers might
want a longer duration than the compute nodes.  (that web cam's not going to
do much for you if the power is shut off...)

A temperature/humidity recorder is nice to have.  It could be as simple as
an off the shelf weatherstation widget and a logging program.  It lets you
better manage your HVAC.

Jim Lux