[Beowulf] Interesting google server design

[Tiago Marques]

On Sat, Apr 4, 2009 at 8:23 AM, Bill Broadley <bill at cse.ucdavis.edu> wrote:

> Robert G. Brown wrote:
> > On Fri, 3 Apr 2009, Greg Lindahl wrote:
> >
> >> On Fri, Apr 03, 2009 at 09:14:37AM -0400, Robert G. Brown wrote:
> >>
> >>>   b) The idea is to get the heat production OFF the motherboard.  One
> >>> really interesting thing about the google design is that they hang the
> >>> stock OTC power supply off the back, altogether outside of the case
> >>
> >> It looks to me like 100% of the airflow through the case goes through
> >> the PS, and in fact the PS fan is the only one driving front-to-back
> >> flow.
> >
> > As it is in a lot of PC cases, but I can't help but think that having
> > the PS in the case itself does raise the temperature in there because it
> > itself gets warm and is in contact with the air in the dead air pocket
> > underneath the PS bracket.
>
> Most normal cases I've seen have a power supply in the top rear and suck
> air
> from the bottom of the power supply (usually very near the cpu) and exhaust
> it
> out the back.  So while it burns about 10% of the energy for the PC
> (typical
> efficiencies in the nicer power supplies are around 90%) very little of
> that
> heat should flow upstream.... unless you lock your PC in a cabinet.  For
> this
> reason I've found it quite strange that high end cases (apple, antex,
> lian-li,
> and others) have put the power supply in a separate airflow chamber in the
> bottom of the case.  Seems of no benefit to the CPU/GPU/motherboard where
> most
> of the heat is generated.


It's of benefit for the PSU and it's done on purpose. It's done to extend
the lifespan of the PSU's components and to make it run cooler and more
silently. In high-end systems with kW PSUs you have a lot of heat generated
by the PSU itself, which when pulling hot air from 2 or 3 graphics cards and
an overclocked CPU starts having an intake not of 20-25ºC but of 35-40ºC. No
PSU can handle this kind of temperature silently. It's manageable in winter
but a problem in summer, when PSUs should not intake air at more than 50ºC
or it will exhibit sub-optimal power delivery.
Also, the separate air-channel usually serves HDDs and PSU, which go very
well together.

Best regards,
                          Tiago Marques



>  I guess it helps the power supply since it's
> sucking air at ambient, or past a few drives which shouldn't change the air
> temp much.
>
> > I was trying to figure out if I "liked" having the battery up front so
> > that air flowed over it before reaching the motherboard, as batteries
> > heat too when they're being charged
>
> Indeed, although if it's fed 12V I'd expect once charged (i.e. 99.9% of
> it's
> life) that the heat would be minimal.  Sure you have a heat spike after a 3
> minute outage.
>
> > but then I remembered that battery
> > lifetime is pretty strongly tied to temperature, so perhaps it makes
> > sense to keep it as cool as anything in the system.
>
> Indeed, it wasn't clear to me if the power supply was the only fan of if
> the
> racks themselves had additional fans in them.  If the powersupply is the
> only
> fan then at least the fan would be running during an outage.  Not too sure
> about the chemistry, but I suspect a discharge in 5 minutes would be quite
> a
> bit hotter than a 30-60 minute charge that seems commonly required to get
> back
> to 90% capacity or so.
>
> > The disk stack did look like it might get a bit hot; I'm guessing that
> > Google disks are pretty hammered.
>
> Disks are only what 5-10 watts each these days?  Doesn't seem like you need
> much air relative to the rest of the system.  Also I believe Google
> published
> some disk data on some n*10k drives over a few years of use and basically
> found no correlation between temperature (within normal usage limits) and
> rate
> of failures.  While not great the google design certainly seems better than
> many hotswap bays I've seen that often mostly enclose the drive (sometimes
> in
> plastic even *ugh*), then put it in a tiny enclosure with a tiny air holes
> drilled in the backplane.  Fortunately the migration from SCA and IDE to
> SAS
> and SATA seem to have helped this a good bit with smaller cables/pinouts.
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20090404/54f67bea/attachment.html>