[Beowulf] Academic sites: who pays for the electricity?

Fri Feb 18 08:36:44 PST 2005

At 17:01 17-2-2005 -0800, Jim Lux wrote:
>At 04:19 PM 2/17/2005, Jim Lux wrote:
>>At 02:54 PM 2/17/2005, David Mathog wrote:
>>>At Wed, 16 Feb 2005 19:08:05 +0100 Vincent Diepeveen wrote:
>>>
>>> > Date: Wed, 16 Feb 2005 19:08:05 +0100
>>> > From: Vincent Diepeveen <diep at xs4all.nl>
>>> > Subject: Re: [Beowulf] Academic sites: who pays for the electricity?
>>> > To: "David Mathog" <mathog at mendel.bio.caltech.edu>,
>>> >       beowulf at beowulf.org
>>> > Message-ID: <3.0.32.20050216190804.0106fcc0 at pop.xs4all.nl>
>>> > Content-Type: text/plain; charset="us-ascii"
>>> >
>>> > At 08:16 16-2-2005 -0800, David Mathog wrote:
>>> > >In most universities services like electricity, water, and
>>> > >A/C are paid for by the school.  To do so they take "overhead"
>>> > >out of every grant.  Partially as a consequence of this they
>>> > >typically have a very poor ability to meter usage on a room
>>> > >by room basis.
>>> > >
>>> > >Now somewhere between the 10 node Pentium II beowulf sitting on
>>> > >a lab bench and the 1000 node dual P4 Xeon beowulf in a machine
>>> > >room that takes up half the basement the cost of the electricity
>>> > >(both for power and A/C) goes from  a minor expense to a major
>>> > >one.  Really major. For instance, in that hypothetical large machine,
>>> > >at 10 cents per kilowatt hour (a round number), assuming 100 watts
>>> > >per CPU (another round number) that's:
>>> > >
>>> > >  1000  (nodes) *
>>> > >     2  (cpus/node) *
>>> > >     .1 (kilowatts/cpu) *
>>> > >     .1 (dollars/kilowatt-hour) *
>>> > >  365   (days /year) *
>>> > >   24   (hours/day) =
>>> > >-----------------------
>>> > >  175200 dollars/year
>>> >
>>> > Complete academic nonsense calculation. If you use quite some
electricity
>>> > the electricity gets up to factor 20-40 cheaper. Getting a factor 10
>>> > reduction in usage bill is pretty easy if you negotiate properly.
>
>Just where do you live that such negotiations are possible. Here's some 

You aren't going to negotiate about a single small room with a few
lightbulbs obviously.

We're talking about huge usage, like if all supercomputers are located at 1
central spot and the entire institute with thousands of working places gets
powered in a central way, and usually electricity offering companies aren't
going to put online their rates for reduced usage, as that would give them
a bad negotation starting point :) 

Nuclear power gets more and more exported from France to rest of Europe.

For example Italy is importing 25% of its total power, majority is from
France. Netherland and Germany import roughly 20% of their power. 

That will get more and more, simply because building electricity producing
central plants can only get build for a specific amount of time (like 25
year contract) and then must get cleaned up. Obviously such rules make
building your own producing central plants impossible for the electricity
producing companies.

I'm not taking a political viewpoint on what type of electricity is
damaging more than the other and what should happen in the future in that
respect. 

Yet closing eyes for reality is something else. More and more power gets
used. Computers take a part of that power, industry majority.

In USA last so many years no new nuclear plants have been built. Obviously
that means that the market there is different from Europe, where the energy
market seems to be more innovative than USA.

Even though for certain plans i have little respect for. Like the 150 meter
high windmills they want to build in Houten, just a few hundreds of meters
away from newly build houses, where tens of thousands live, that is IMHO a
wrong idea. 

Don't have the details here how big the diameter and speed of airtransport
is of those mills, but obviously they can only get build because the
government wastes money on them and at most kill huge number of birds who
have near zero chance to survive if they are near those mills. 

Yet it's another innovation in Europe.

The energy market is one of the most complex financial markets and not only
because politicians prefer to close their eyes for its problems. Another
major problem is who owns what in europe. Is the government again going to
own the transport infrastructure or can independant transport companies
keep doing the job? There is difference between energy producers and energy
consumer delivering companies and so on.

Yet a washing machine eats thousands of watts and nearly everybody has one
at home in Europe. Certain products we daily use and just throw away get
produced by throwing tens of thousands of watts into battle in heavy
industry. So complaining about energy usage of computers is a good thing,
but shouldn't get overreacted.

>real numbers from Southern California Edison. 
>http://www.sce.com/CustomerService/RateInformation/BusinessRates/LargeBusin
ess/
>
>First off, you're looking at a 200kW load for 1000 nodes, which is a hefty 
>load, just for the computers (not counting lights, HVAC, etc.)  But, no 
>matter, we'll assume your facility is sucking at least 500kW some of the 
>time, so that would put you in the large business TOU-8 tariff.
>http://www.sce.com/NR/sc3/tm2/pdf/ce54-12.pdf
>  All the large consumer tariffs are time-of-use sensitive.  I assume you 
>wouldn't want some sort of "Critical Peak Pricing Options" or "Demand 
>Bidding Programs"
>
>Let's assume you're being served at 240V (as opposed to having your own 
>distribution transformers, etc., although as a 200kW consumer, that's 
>something you should consider).
>
>Looks like the rates break down as about 0.016/kWh for the delivery, and 
>the actual power (generation) runs somewhere between 0.04/kWh  (off peak 
>summer) to 0.12/kWh on peak summer.
>
>There's also a raft of other charges (metering, demand (runs about $10/kW 
>of instantaneous demand), power factor, etc.)
>
>Compare this to Domestic service.. where the rates run from 0.11 to 
>0.18/kWh, depending on where you sit relative to baseline, season, etc. 
>(I'll also point out that I was paying SCE $0.26/kWh at home in the summer 
>of 2001, but rates are lower now.) Now, you might consider Residential to 
>be artificially constrained for political reasons, so we take a look at 
>GS-1 (general service..)
>Here, we have 0.07 for delivery and 0.085 (totalling 0.155/kWh) during the 
>summer.
>
>The point is, there isn't a 10:1 ratio... not even close. And, rgb's 
>ballpark of $0.10/kWh is a perfectly reasonable estimate, if a bit low for 
>Southern California. Even as far back as 1990,on peak, large customers were 
>paying on the order of $0.11/kWh.
>
>It is possible that if you were buying power directly (which is possible, 
>as a large end user), you can pay "market rate" for each kWh you 
>consume.  The price is quite volatile, though... At the peak of the market 
>failure a few years back, a kWh on the open market was something like $20 
>during peak times.  I doubt you want to schedule your cluster ops to take 
>advantage of electricity rate fluctuations.
>
>
>>>Well, it isn't complete nonsense, unless you care to dispute the
>>>number of days in a year, hours in a day, or cpus in a dual node
>>>computer!
>>>
>>>The only term you're complaining about is the price of
>>>electricity.  I'm not privy to the electrical rates that our
>>>school pays, they may well be an order of magnitude lower.  My
>>>home rates certainly aren't, but then, I don't buy as much
>>>power as the campus.  It's also not at all clear that the
>>>campus would sell power to the end users at the same rate
>>>which it pays the utility.
>
>CalTech probably buys their power from City of Pasadena, but it's probably 
>similar in rate structure to SCE's TOU-8.  Home rates are somewhat 
>artificially low for political reasons.  The folks really getting the short 
>end of the stick are small businesses who don't have the negotiating power 
>that a large business does, nor the political clout of elderly pensioners 
>dying from heat.
>
>
>I'll also note that if you start paying for kWh, you're going to want to 
>give serious consideration to buying more nodes than you strictly need, and 
>shutting down the cluster during on-peak times. A typical pricing strategy 
>might be 0.15/0.07/0.05 (on/mid/off peak): Peak lasts 6 hrs (1200-1800), 
>mid is 0800-1200,1800-2300, and offpeak is the rest.  There's 93 hours of 
>offpeak time out of 168 total in the week
>
>For the pricing and schedules above, it turns out that the optimum is to 
>shut down only during peak, but run during midpeak, for an average energy 
>cost of 0.056/kWh, but using 22% more computers.
>
>
>
>James Lux, P.E.
>Spacecraft Radio Frequency Subsystems Group
>Flight Communications Systems Section
>Jet Propulsion Laboratory, Mail Stop 161-213
>4800 Oak Grove Drive
>Pasadena CA 91109
>tel: (818)354-2075
>fax: (818)393-6875
>
>
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
>
>