[Beowulf] Rack power requirement question

[Jim Lux]

At 11:35 PM 4/14/2005, David Kewley wrote:
>Hi David,
>
>David Mathog wrote on Thursday 14 April 2005 11:59:
> > For the design of a new computer room I need to put together an
> > estimate ASAP of typical power usage by a mythical
> > "full computer rack" - 3 years from now. My crystal ball is a bit
> > fuzzy in that range
>
>I wonder who has a crystal ball that *isn't* fuzzy in this realm?
>
> > so I thought it best to collect several
> > opinions and then present an answer based on those values. If you
> > have a moment please fill in your best guess for power
> > consumption values, in 2008, in kilowatts for:
> >
> > 1.  a "high end" rack               (most power)
> > 2.  an "average" rack               (average power)
> > 3.  the "minimum worthwhile" rack   (least power)
> >
> > The third one is a probably not named well, so here's an example
> > in today's terms: this rack might be stocked with Pentium III's
> > from 2001 or so, since anything cooler than that wouldn't
> > calculate fast enough to be worth the effort of maintaining it.
>
>The only reasonably easy one, to me at least, is #3. :)  I'd say 10kW,
>assuming around 40 nodes at 250W apiece.  If you don't like 250W/node
>increase accordingly e.g. to 12kW.
>
>Guessing 3 years out?  I don't think the CPU manufacturers themselves have a
>good idea what the power consumption will be.  Will processors keep burning
>more power, or will engineers more squarely face the challenge of reducing
>power or better managing it?  Will future 1U boxes have typically 2
>processors or 4?  Will CPUs be quad-core?  Will 1U's be out & blades in?
>Will blades be out & multi-U supercomputer-in-a-box be in?  Will the Cell
>processor and similar architectures hit mainstream and turn everything on its
>head?
>
>As an example of how invalid simple extrapolation is, Intel will be selling
>around the end of this year a dual-core Itanium (Montecito) that has 1.7x10^9
>transistors and 24MB of L3 cache.  It runs faster than current Itaniums, yet
>consumes less power (not less than 2x, less than 1x!).
>
>I'd guess that any one CPU will be limited around 100W.

This is probably true.  Since they're selling into a mass market, thermal 
management is an issue, and it's hard to get the heat out of higher power 
density devices. (We've got some kilowatt RF power FETs down in the lab 
that have this problem in a big way).

Maybe a way to do it is to figure on a power density (watts/cubic cm) 
basis, which tends to remain fairly constant.  3W/cm3 might be a decent 
target number?

Power converter densities can run up to 50 W/in3 -- also around 3W/cm3


>  I could imagine 4
>CPUs in a 1U box,

Now here is where you could have a substantial density increase, 
though.  With higher integration (so you don't spread everything out on a 
mobo), you could get higher densities.

However, let's just assume that your overall volumetric power density is 
around 1W/cm3 (CPU and power converter together is already 1.5W/cm3)

A rack is say, 2m tall, 40 cm wide and 40 cm deep.... 320kW per rack is 
probably a reasonable upper bound (in that no matter how exotic they get, 
they're not going to go past that).  Compare to 42 1U 4 processor widgets 
at 800 W each which is about a tenth of that.

If they're at 30kW/rack now, I think it would plausible that you might get 
to 100kW/rack sometime in the future.  (however, in terms of "room volume" 
you might not get much more dense than the current... that 100kW/rack will 
probably need several racks worth of specialized equipment to support it (a 
100kVA breaker panel is about the size of a rack.. as is a 100kVA 
distribution transformer)



>add another 100W for everything else, and you're up around
>500W.  Times 40U, you get 20kW.  I guess blades are similar even now.
>
>Maybe the high end rack in 2008 will be 25kW?
>
>David
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit 
>http://www.beowulf.org/mailman/listinfo/beowulf

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875