[Beowulf] Interesting google server design

[Bill Broadley]

Robert G. Brown wrote:
> On Fri, 3 Apr 2009, Greg Lindahl wrote:
> 
>> On Fri, Apr 03, 2009 at 09:14:37AM -0400, Robert G. Brown wrote:
>>
>>>   b) The idea is to get the heat production OFF the motherboard.  One
>>> really interesting thing about the google design is that they hang the
>>> stock OTC power supply off the back, altogether outside of the case
>>
>> It looks to me like 100% of the airflow through the case goes through
>> the PS, and in fact the PS fan is the only one driving front-to-back
>> flow.
> 
> As it is in a lot of PC cases, but I can't help but think that having
> the PS in the case itself does raise the temperature in there because it
> itself gets warm and is in contact with the air in the dead air pocket
> underneath the PS bracket.

Most normal cases I've seen have a power supply in the top rear and suck air
from the bottom of the power supply (usually very near the cpu) and exhaust it
out the back.  So while it burns about 10% of the energy for the PC (typical
efficiencies in the nicer power supplies are around 90%) very little of that
heat should flow upstream.... unless you lock your PC in a cabinet.  For this
reason I've found it quite strange that high end cases (apple, antex, lian-li,
and others) have put the power supply in a separate airflow chamber in the
bottom of the case.  Seems of no benefit to the CPU/GPU/motherboard where most
of the heat is generated.  I guess it helps the power supply since it's
sucking air at ambient, or past a few drives which shouldn't change the air
temp much.

> I was trying to figure out if I "liked" having the battery up front so
> that air flowed over it before reaching the motherboard, as batteries
> heat too when they're being charged

Indeed, although if it's fed 12V I'd expect once charged (i.e. 99.9% of it's
life) that the heat would be minimal.  Sure you have a heat spike after a 3
minute outage.

> but then I remembered that battery
> lifetime is pretty strongly tied to temperature, so perhaps it makes
> sense to keep it as cool as anything in the system.

Indeed, it wasn't clear to me if the power supply was the only fan of if the
racks themselves had additional fans in them.  If the powersupply is the only
fan then at least the fan would be running during an outage.  Not too sure
about the chemistry, but I suspect a discharge in 5 minutes would be quite a
bit hotter than a 30-60 minute charge that seems commonly required to get back
to 90% capacity or so.

> The disk stack did look like it might get a bit hot; I'm guessing that
> Google disks are pretty hammered.

Disks are only what 5-10 watts each these days?  Doesn't seem like you need
much air relative to the rest of the system.  Also I believe Google published
some disk data on some n*10k drives over a few years of use and basically
found no correlation between temperature (within normal usage limits) and rate
of failures.  While not great the google design certainly seems better than
many hotswap bays I've seen that often mostly enclose the drive (sometimes in
plastic even *ugh*), then put it in a tiny enclosure with a tiny air holes
drilled in the backplane.  Fortunately the migration from SCA and IDE to SAS
and SATA seem to have helped this a good bit with smaller cables/pinouts.