[Beowulf] Interesting google server design

[Lux, James P]

> -----Original Message-----
> From: Donald Becker [mailto:becker at scyld.com] 
> Sent: Thursday, April 02, 2009 11:36 AM
> To: Lux, James P
> Cc: Greg Lindahl; beowulf at beowulf.org
> Subject: RE: [Beowulf] Interesting google server design
> 
> On Thu, 2 Apr 2009, Lux, James P wrote:
> > > On Thu, Apr 02, 2009 at 10:11:07AM -0400, Prentice Bisbal wrote:
> > > > Or it could be because they make motherboards that convert
> > > 12 VDC to 5
> > > > VDC on the motherboard.
> > > Greg Lindahl wrote: 
> > > All Itanium and some other x86 boxes take a single 48V 
> input to the 
> > > mobo. I talked to a mobo designer once and he claimed 
> that there was 
> > > no power savings to be had doing this. Beats me, mon.
>  
> > If I were king of the world and could dictate designs, 
> here's what I would do:
> > 
> > A big bulk 3 phase rectifier to make the 400V bus, which would be 
> > fairly
> > 
> > The batteries hang off this bus at that voltage. (this is standard 
> > stuff these days, with grid-tie inverters, for instance).
> 
> That leads to short battery life.  If you take this approach 
> you really want the intermediate voltage to be optimized to 
> the battery float voltage.  This is a century-old design used 
> with 48V telephone circuits and giant lead-acid cells in the 
> basement, but using it with higher voltages and better 
> batteries doesn't work well.

Why would the life change? The voltage per cell is identical whether you use 24 cells for 48V nominal or 150 cells for 300V nominal.  The lead acid battery life is almost entirely determined by temperature and charge/discharge current profiles, which are presumed to be appropriately controlled.

I'm not proposing that one uses a trickle/float charge type architecture.. If you're on a multitens of kilowatt scale (where three phase power and a non-standard power scheme in general starts to be worth considering), you'd set up the system appropriately.  For instance, the batteries here are not necessarily floated across the bus, but can supply the bus voltage in an outage (and you'd choose batteries with a current output capability appropriate for the time you'll be operating.. Maybe 20 minutes or whatever it takes to get your standby generator fired up)  This is an application more like a giant UPS than a telco trickle charge.




>  
> > You distribute the 400VDC (or 350, or whatever it works out to be
> > convenient) to all the chassis.
> 
> Here is where you run into problems.  If you use 
> telephony-standard 48V, you need really hefty power cables.  
> If you use high voltage, you need better insulation and 
> special precautions.  If you run 400VDC to the server boards, 
> you need special precautions for the power connections and 
> everything connected to the board.  Anywhere it's possible 
> for the 400VDC to get to needs to be shielded from human 
> contact and isolated from the rest of the system. 

Which is no different, really, than what is done now for the 240VAC line current, eh?  There ARE some issues with DC vs AC, but overall, once you've bought into "custom" at all, you'd trade off those sorts of things. 


 A FET 
> shorting 400VDC through a heatsink or to a ground plane can 
> fry a whole board, and then the human that goes to replace it.
> 
> Suddenly running 120V or 240V to isolated metal-boxed power 
> supplies looks like a great engineering solution.

Really no different from running 350V DC to isolated metal-boxed power supplies (since 240V is really 336V peak, etc.).  There are off-the-shelf solutions to all this sort of thing.  Maybe not in consumer products, but still in high volume production, so costs are low.


> 
> > Because you're feeding from an essentially regulated DC 
> bus, the "per 
> > PC" power supply can be designed with higher efficiency over a 
> > narrower input voltage range (2-3% should be easy to achieve, with 
> > fairly
> 
> DC-to-DC converters can be 98% efficient (although 94% is 
> typical), but they lose efficency when they have to convert a 
> high ratio. 

Only if you don't have a transformer. If you're doing a transformerless "buck" converter, yes, a big drop will have short duty cycle, so switching losses will be large.  However, nobody would do a straight buck converter in this application (for one thing, there's no primary/secondary isolation).  

You'd basically take a standard PC power supply design and get rid of the input rectifier and filter.  The ability to have a somewhat regulated DC bus lets you optimize the design.



 To drop to 48V to 3.3V you'll want at least one 
> intermediate stage.. say 12V.  And if we are close, we might 
> as well regulate it well and use it for the disk drives.  
> Hmmm, aren't we back at pretty much the standard design?



> 
> -- 
> Donald Becker				becker at scyld.com
> Penguin Computing / Scyld Software
> www.penguincomputing.com		www.scyld.com
> Annapolis MD and San Francisco CA
> 
>