[Beowulf] A/C to water cooling, considering the transition

[Andrew M.A. Cater]

On Thu, Apr 14, 2005 at 02:21:48PM -0600, Josip Loncaric wrote:
> David Mathog wrote:
> >
> >Or is chilled water  too cold for direct use in cooling
> >computer equipment?  (For nstance, causing condensation
> >problems.)
> 
Today's air cooled equipment is potentially/actually 
at the same power density and heat output as the Crays cooled with flourinert 
or the big water cooled mainframes were a few years ago.  Price has dropped, 
processing power has improved - but so has the need for cooling. 

IMHO and without much direct experience. Use cooled water for
chilled beams and aircon. and cool the equipment that way.

If you need direct liquid cooling of rack parts / servers in the rack
you've a whole host more problems, not least the fact that liquid and
electricity don't mix and that air cooled units have holes in them.
There _will_ be liquid coolant leaks eventually.  Old style
supercomputers were built on an "island" - if coolant leaked, there were
dams to catch it and electricity came in above the level of the coolant
overflows.

Watch out for condensation on beams - make sure they are (a). the right
way up (b) have drip trays underneath. [Fond memories of a lab. where,
potentially, neither of these were true. Switch the chilled beams up and
you'd get indoor rain from condensation :( As a result, I lived at 30-35
C all year, winter and summer]

Big Peltier effect plates might help a lot - but they're almost as
expenisve as the supercomputers if you want something to produce very
meaningful cooling :(



> I'm not sure that you'd be using pure chilled water to cool your CPUs 
> directly.  Apple's liquid cooled Mac uses some kind of alcohol, I think.
> 
> Since air cooling dominates today, there are also transitional 
> technologies to consider.  A mixed air/water cooling scenario involves a 
> water-cooled rear door to cool the hot air coming out of a rack; this 
> doesn't cause condensation if that the liquid is not too cold.
> 
> See http://www.designnews.com/article/CA500772.html for an example: 
> Altix systems can dissipate ~8KW per rack, so they use a water cooled 
> rear door, which works with chilled liquid of just under 60 deg. F. 
> Just in case any condensation happens anyway, they added a drain feature 
> to the rack.
> 
> Given the rapid increases in packaging density (e.g. blade servers with 
> power densities much higher than SGI's Altix), I'm pretty sure that 
> large computer installations will want liquid cooling.  Making this 
> transition is a challenge today, but within a few years, more plumbing 
> will be needed in high end computer rooms.
> 

I walked into someone else's computer rooms the other day, having been
invited to see the kit.  Noisy - but virtually empty. However, all the
power budget has been allocated and all the space will be taken up.
Designed at least 6 - 8 years ago and based on best available guesswork
at the time: no room for expansion or increased power needs now. 

Building a reasonable size computer hall takes a couple of
years of planning and another couple of years to build - so the sysadmin 
needs to explain that they're really full, even though the hall looks 
3/4 empty today, if they're going to plug the shortfalls in a couple of years.
Not the best way to impress your funding authorities :(

Andy