[Beowulf] 'liquid cooled' racks

[Bruce Allen]

Dear Beowulf list,

For my next cluster room, I am hoping to use 'liquid cooled' racks make by 
Knurr (CoolTherm, http://www.thermalmanagement.de/).  The scale is 67 
racks x 7.5kW heat removal per rack (36 U usable per rack).

Does anyone on the list have experiences with these or similar racks (you 
can reply to me privately if you don't want to share your experiences 
publicly)?  I am told that there are lots of installations in Europe, 
especially in Germany, but that the only US sites are at Penn State, 
somewhere in the Chicago area, and U. of Vermont, and that these US 
installations are fairly small-scale.

The racks have front and back doors that close, and contains fans in the 
back which circulate air through a heat exchanger located in the bottom of 
the rack.  The heat exchanger transfers the heat into chilled water.

The advantages of this are that it is quiet, and that you don't need 
vertical height for underfloor ducting or overhead hot air removal. 
Disadvantages are cost, potential difficulty of working within the rack. 
and loss of one rack of cooling capacity if you open both the front and 
back doors.

(In this case I have access to 'building' funds that can not be used to 
buy more cpus, so the cost issue is not important.)

I would be very interested to hear about people's experiences with these 
racks (or similar ones from another manufacturer) in an HPC environment.

Cheers,
 	Bruce