[Beowulf] Opteron cooling specifications?

[Michael Will]

A good set of specs according to our engineers could be:

1. No side vending of hot air from the case. The systems will be placed
into 19" racks and there is no place for the air to go if it's blown
into the side of the rack. Even if you take the sides off then you still
will have racks placed next to each other. Airflow should be 100% front
to back.

2. Along with that, there should be no "cheat holes" in the top, bottom
or sides of the case. All "fresh" air should be drawn in from the front
of the chassis. Again, the system will be racked in a 19" rack and there
is no "fresh air" to be drawn in from the sides of the case (see 1
above) nor will the holes be open when nodes are stacked one on top of
the other in a fully populated rack (32 nodes per rack).

3. There should be a mechanical separation between the hot and cold
sections of the chassis to prevent the internal fans from sucking in hot
air from the rear of the chassis.

4. The power supply *must* vent directly to the outside of the case and
not inside the chassis. The power supply produces approximately 20% of
the heat in the system. That hot air must be vented directly out of the
chassis to prevent it from heating other components in the system.

5. The system should employ fan speed control. Running high speed fans
at less than rated speed prolongs their life and reduces power usage for
the platform as a whole. Fan speed should be controlled by either
ambient temperature or preferably by CPU temperature.

6. The system must have a way of measuring fan speed and reporting a fan
failure so that failed fans can be replaced quickly.
 
Michael Will / SE Technical Lead / Penguin Computing

-----Original Message-----
From: beowulf-bounces at beowulf.org [mailto:beowulf-bounces at beowulf.org]
On Behalf Of Bill Broadley
Sent: Thursday, April 27, 2006 2:55 PM
To: beowulf at beowulf.org
Subject: [Beowulf] Opteron cooling specifications?


I'm writing a spec for future opteron cluster purchases.  The issue of
airflow came up.

I've seen a surprising variety of configurations, some with a giant
rotating cylinder (think paddle wheel), most with a variety of 40x28 or
40x56mm fans, or horizontal blowers.

Anyone have a fan vendor they prefer?  Ideally well known for making
fans that last 3+ years when in use 24/7.

A target node CFM for a dual socket dual core opteron?

A target maximum CPU temp?  I assume it's wise to stay well below the
70C or so thermal max on most of the dual core Opterons.

Seems like there is a huge variation in the number of fans and total CFM
from various chassis/node manufacturers.  A single core single socket 1u
opteron I got from sun has two 40mm x 56mm, and 4 * 40mm x 28mm fans.
Not bad for a node starting at $750.

Additionally some chassis designs form a fairly decent wall across the
node for the fans to insure a good front to back airflow.  Others seem
to place fans willy nilly, I've even seen some that suck air sideways
across the rear opteron.

In any case, the nature of the campus purchasing process is that we can
put in any specification, but can't buy from a single vendor, or award
bids for better engineering.  So basically lowest bid wins that meets
the spec.  Thus the need for a better spec. 

Any feedback appreciated.

--
Bill Broadley
Computational Science and Engineering
UC Davis
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org To change your subscription
(digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf