[Beowulf] cooling question: cfm per rack?

[Mark Hahn]

> The second term really has two parts: 
>   A.  the amount of air moved
>   B.  the reduction in temperature of that air across the A/C unit
> 
> The latter part is specified in tons.  The A/C guys I've spoken

well, I usually think of temperature as a side-effect of the more 
direct measure, movement of energy.  hence, I always think of the 
tidy relation of 3.517 KW = 1 ton.  I usually skip any BTUs...

> with recently utilize some more or less standard relationship
> between cubic feet per minute (cfm) and A/C tons for the units they
> maintain.

CFM and delta-t across the machine-to-be-cooled are convolved to give 
you how much heat you're extracting.  no doubt both pressure and humidity
are involved to some degree as well, and I don't have a good equation
for this.  the good thing is that turning down the temperature can partly
mitigate minor airflow problems.

some reasonable discussion from Intel, (a bit axe-grinding, though):
http://www.7x24nw.org/Presentations-folder/Air%20Cooling%20in%20Servers%20and%20IT%20Facilities.pdf

a dell 1855 blade chassis spec's 400 CFM for ~4KW.  they're talking 6 of 
those chassis in a rack (24 KW!).  then again, that's assuming an unrealistic
power-per blade (>400W), which sounds like corporate CYA to me:
http://www.dell.com/downloads/global/products/pedge/en/PowerEdge%201855%20DC%20Whitepaper.pdf

this is a good overall discussion, though perhaps a bit pessimistic
about "typical" machinerooms:
http://www.chatsworth.com/uploads/pdf/best_practices_cooling_wp.pdf
http://www.chatsworth.com/uploads/pdf/increase_computerrm_cooling_wp.pdf

sun recommends 21-23C, 45-50%.  35% min, ESD critical at 30%:
http://www.sun.com/products-n-solutions/hardware/docs/html/817-4137-10/2__EnvReq.html

to complicate matters, HVAC folk always bring up the issue of "sensible
load".  as near as I can tell, this is just a way of saying that if you try
to impose too much delta-T on humid air, you wind up wasting a lot of energy
dehumidifying it...

tiles between 500-2000 CFM:
http://h200005.www2.hp.com/bc/docs/support/SupportManual/c00064724/c00064724.pdf
that also gives:
	CFM = btu/hr / (1.08 * dT)
so for 1 ton = 12000 BTU/hr and 70->90, 555 CFM per ton of cooling.
HVAC folk also tend to say 1 tile/ton, which seems about right.

> These run off the campus cold water supply, so
> it makes sense that heat out is proportional to flow across, assuming
> that the cold water has a very large heat capacity.

our experience with CW has been disasterous, but we made the huge mistake
of not using precision/machineroom chillers (fancoils, actually).
our old/existing machineroom, for instance, is supposed to have 2x8ton
fancoils, but combined they never moved more than about 20 KW (should be 56).

unless you have pretty extreme assurances about WC quality (flow, temp),
I would only consider using dual-cool machineroom chillers (DX + CW, usually
adds about 15% to price.)

> directly through the A/C.  Even more ideally cfm through _each_ rack
> could be modulated somehow, since some racks move much more 
> air than others and putting a low flow rack next to a high flow rack
> might drive the air the wrong way through the low flow unit.

well, the stuff in racks does probably have quite a few fans,
which could ideally modulate themselves.  my current-gen clusters 
certainly don't do that, but I'd be quite happy if next-gen did...

> How does one calculate an optimal cfm through a rack? 
> 
> For a specific example with round numbers, let's say it's a
> 25U rack, dissipates 10kW, and has a single 50 cfm per minute output
> fan per 1U node.  (Ie, all air out must go through that path.)

that sounds reasonable to me - 10KW is ~3 tons, and the formula above
relates your 1250 CFM total to about 3 tons as well.  for my 10KW racks,
I'm hoping to push the temperature down a bit (60-65), keep the humidity
low to avoid "sensible" wastage, and hope for the best with our tiles.

> Is cfm the key unit here or should one think in terms of pressure
> at various points in the room?

I think the answer is yes.  with a good raised floor, you seem to be 
able to expect fairly even pressure distribution.  we turned on our 
new machineroom yesterday, and the pressure feels similar everywhere 
(16" raised floor, though with some conduits down there, and 3x30T
Liebert deluxe system 3's.)

if your pressure is reasonably even, the same tiles should flow the 
same CFM.  I'd LOVE to find some way to measure airflow, since I'd 
actually consider doing things like adding patches of duct tape to 
the underside of too-high-flow tiles.  I suppose that the empiricist
approach is just to sample all your system temperatures, and if some 
are too high, reduce the airflow to racks which are "too cool".