[Beowulf] Academic sites: who pays for the electricity?

[David Mathog]

At Wed, 16 Feb 2005 19:08:05 +0100 Vincent Diepeveen wrote:

> Date: Wed, 16 Feb 2005 19:08:05 +0100
> From: Vincent Diepeveen <diep at xs4all.nl>
> Subject: Re: [Beowulf] Academic sites: who pays for the electricity?
> To: "David Mathog" <mathog at mendel.bio.caltech.edu>,
> 	beowulf at beowulf.org
> Message-ID: <3.0.32.20050216190804.0106fcc0 at pop.xs4all.nl>
> Content-Type: text/plain; charset="us-ascii"
> 
> At 08:16 16-2-2005 -0800, David Mathog wrote:
> >In most universities services like electricity, water, and 
> >A/C are paid for by the school.  To do so they take "overhead"
> >out of every grant.  Partially as a consequence of this they
> >typically have a very poor ability to meter usage on a room
> >by room basis.
> >
> >Now somewhere between the 10 node Pentium II beowulf sitting on
> >a lab bench and the 1000 node dual P4 Xeon beowulf in a machine
> >room that takes up half the basement the cost of the electricity
> >(both for power and A/C) goes from  a minor expense to a major
> >one.  Really major. For instance, in that hypothetical large machine,
> >at 10 cents per kilowatt hour (a round number), assuming 100 watts
> >per CPU (another round number) that's:
> >
> >  1000  (nodes) *
> >     2  (cpus/node) *
> >     .1 (kilowatts/cpu) *
> >     .1 (dollars/kilowatt-hour) *
> >  365   (days /year) *
> >   24   (hours/day) =
> >-----------------------
> >  175200 dollars/year
> 
> Complete academic nonsense calculation. If you use quite some electricity
> the electricity gets up to factor 20-40 cheaper. Getting a factor 10
> reduction in usage bill is pretty easy if you negotiate properly.

Well, it isn't complete nonsense, unless you care to dispute the
number of days in a year, hours in a day, or cpus in a dual node
computer!
  
The only term you're complaining about is the price of
electricity.  I'm not privy to the electrical rates that our
school pays, they may well be an order of magnitude lower.  My
home rates certainly aren't, but then, I don't buy as much
power as the campus.  It's also not at all clear that the
campus would sell power to the end users at the same rate
which it pays the utility.

I don't really understand your point about keeping the units
running versus restarting them.  Sure, it would be really bad
to try to boot all 1000 nodes simultaneously, in all likelihood
it wouldn't work.  That's why they are typically started at N
second intervals, where N depends on your hardware.
Surely there is some N large enough so that the peak current
draw during the restart never exceeds the random fluctuations
observed when all units are running normally.  Or is your
point that the electricity company doesn't want the facility
to draw _less_ current than it uses normally at
steady state?

On a somewhat related note, it would be nice if rack nodes
had some graceful way to conserve electricity.  For instance,
something along the lines of: if the CPU utilization goes
below 5% for 10 seconds ratchet the clock down by a factor of 10.
When CPU usage goes above 90% ratchet for 2 seconds move it back
up again.  Notebooks can do this sort of thing, but it seems not
to be a "feature" of most full size motherboards.  This should
also lower the average temperature in the case, at the expense
of increased thermal cycling.  Hard to say off hand if that's
a plus or a minus as far as hardware longevity goes.  Certainly
it would be a plus in terms of energy conservation.

Regards,

David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech