[Beowulf] Academic sites: who pays for the electricity?

[David Mathog]

In most universities services like electricity, water, and 
A/C are paid for by the school.  To do so they take "overhead"
out of every grant.  Partially as a consequence of this they
typically have a very poor ability to meter usage on a room
by room basis.

Now somewhere between the 10 node Pentium II beowulf sitting on
a lab bench and the 1000 node dual P4 Xeon beowulf in a machine
room that takes up half the basement the cost of the electricity
(both for power and A/C) goes from  a minor expense to a major
one.  Really major. For instance, in that hypothetical large machine,
at 10 cents per kilowatt hour (a round number), assuming 100 watts
per CPU (another round number) that's:

  1000  (nodes) *
     2  (cpus/node) *
     .1 (kilowatts/cpu) *
     .1 (dollars/kilowatt-hour) *
  365   (days /year) *
   24   (hours/day) =
-----------------------
  175200 dollars/year

The A/C expense is going to vary tremendously depending upon
the outside temperature.  It's going to be much higher for us
in Southern California than for a site in Anchorage.

"Typical" lab usage is widely variable but I'd be amazed
if most biology or chemistry labs burn through even 1/10th this
much for the equivalent lab area.  Some physics lab running
a tokamak might come close.


Anyway, the question is, have any of the universities said "enough
is enough" and started charging these electricity costs directly?
If so, what did they use for a cutover level, where usage was
"above and beyond" overhead?

>From an economic perspective having electricity and A/C come out
of overhead (without limit) grossly distorts the true cost
of the project over time and can lead to choices which increase
the total overall cost. For instance, the use of Xeons instead of
Opterons has little effect on TCO if somebody else is picking
up the electricity tab, but could change the power consumption
significantly on a large project.

Regards,

David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech