a tale of two supercomputers
eugen at leitl.org
Tue Jun 25 05:45:53 PDT 2002
June 25, 2002
At Los Alamos, Two Visions of Supercomputing
By GEORGE JOHNSON
Moore's Law holds that the number of transistors on a microprocessor the
brain of a modern computer doubles about every 18 months, causing the
speed of its calculations to soar. But there is a downside to this
oft-repeated tale of technological progress: the heat produced by the chip
also increases exponentially, threatening a self-inflicted meltdown.
A computer owner in Britain recently dramatized the effect by propping a
makeshift dish of aluminum foil above the chip inside his PC and frying an
egg for breakfast. (The feat cooking time 11 minutes was reported in
The Register, a British computer industry publication.) By 2010,
scientists predict, a single chip may hold more than a billion
transistors, shedding 1,000 watts of thermal energy far more heat per
square inch than a nuclear reactor.
The comparison seems particularly apt at Los Alamos National Laboratory in
northern New Mexico, which has two powerful new computers, Q and Green
Destiny. Both achieve high calculating speeds by yoking together webs of
commercially available processors. But while the energy-voracious Q was
designed to be as fast as possible, Green Destiny was built for
efficiency. Side by side, they exemplify two very different visions of the
future of supercomputing.
Los Alamos showed off the machines last month at a ceremony introducing
the laboratory's Nicholas C. Metropolis Center for Modeling and
Simulation. Named for a pioneering mathematician in the Manhattan Project,
the three-story, 303,000-square-foot structure was built to house Q, which
will be one of the world's two largest computers (the other is in Japan).
Visitors approaching the imposing structure might mistake it for a power
generating plant, its row of cooling towers spewing the heat of
computation into the sky.
Supercomputing is an energy-intensive process, and Q (the name is meant to
evoke both the dimension-hopping Star Trek alien and the gadget-making
wizard in the James Bond thrillers) is rated at 30 teraops, meaning that
it can perform as many as 30 trillion calculations a second. (The measure
of choice used to be the teraflop, for "trillion floating-point
operations," but no one wants to think of a supercomputer as flopping
trillions of times a second.)
Armed with all this computing power, Q's keepers plan to take on what for
the Energy Department, anyway, is the Holy Grail of supercomputing: a
full-scale, three-dimensional simulation of the physics involved in a
"Obviously with the various treaties and rules and regulations, we can't
set one of these off anymore," said Chris Kemper, deputy leader of the
laboratory's computing, communications and networking division. "In the
past we could test in Nevada and see if theory matched reality. Now we
have do to it with simulations."
While decidedly more benign than a real explosion, Q's artificial blasts
described as testing "in silico" have their own environmental impact.
When fully up and running later this year, the computer, which will occupy
half an acre of floor space, will draw three megawatts of electricity. Two
more megawatts will be consumed by its cooling system. Together, that is
enough to provide energy for 5,000 homes.
And that is just the beginning. Next in line for Los Alamos is a
100-teraops machine. To satisfy its needs, the Metropolis center can be
upgraded to provide as much as 30 megawatts enough to power a small
That is where Green Destiny comes in. While Q was attracting most of the
attention, researchers from a project called Supercomputing in Small
Spaces gathered nearby in a cramped, stuffy warehouse to show off their
own machine a compact, energy-efficient computer whose processors do not
even require a cooling fan.
With a name that sounds like an air freshener or an environmental group
(actually it's taken from the mighty sword in "Crouching Tiger, Hidden
Dragon"), Green Destiny measures about two by three feet and stands six
and a half feet high, the size of a refrigerator.
Capable of a mere 160 gigaops (billions of operations a second), the
machine is no match for Q. But in computational bang for the buck, Green
Destiny wins hands down. Though Q will be almost 200 times as fast, it
will cost 640 times as much $215 million, compared with $335,000 for
Green Destiny. And that does not count housing expenses the $93 million
Metropolis center that provides the temperature-controlled, dust-free
environment Q demands.
Green Destiny is not so picky. It hums away contentedly next to piles of
cardboard boxes and computer parts. More important, while Q and its
cooling system will consume five megawatts of electrical power, Green
Destiny draws just a thousandth of that five kilowatts. Even if it were
expanded, as it theoretically could be, to make a 30-teraops machine
(picture a hotel meeting room crammed full of refrigerators), it would
still draw only about a megawatt.
"Bigger and faster machines simply aren't good enough anymore," said Dr.
Wu-Chung Feng, the leader of the project. The time has come, he said, to
question the doctrine of "performance at any cost."
The issue is not just ecological. The more power a computer consumes, the
hotter it gets. Raise the operating temperature 18 degrees Fahrenheit, Dr.
Feng said, and the reliability is cut in half. Pushing the extremes of
calculational speed, Q is expected to run in sprints for just a few hours
before it requires rebooting. A smaller version of Green Destiny, called
Metablade, has been operating in the warehouse since last fall, requiring
no special attention.
"There are two paths now for supercomputing," Dr. Feng said. "While
technically feasible, following Moore's Law may be the wrong way to go
with respect to reliability, efficiency of power use and efficiency of
space. We're not saying this is a replacement for a machine like Q but
that we need to look in this direction."
The heat problem is nothing new. In taking computation to the limit,
scientists constantly consider the trade-off between speed and efficiency.
I.B.M.'s Blue Gene project, for example, is working on energy-efficient
supercomputers to run simulations in molecular biology and other sciences.
"All of us who are in this game are busy learning how to run these big
machines," said Dr. Mike Levine, a scientific director at the Pittsburgh
Supercomputing Center and a physics professor at Carnegie Mellon
University. A project like Green Destiny is "a good way to get people's
attention," he said, "but it is only the first step in solving the
Green Destiny belongs to a class of makeshift supercomputers called
Beowulf clusters. Named for the monster-slaying hero in the eighth-century
Old English epic, the machines are made by stringing together
off-the-shelf PC's into networks, generally communicating via Ethernet
the same technology used in home and office networking. What results is
supercomputing for the masses or, in any case, for those whose operating
budgets are in the range of tens or hundreds of thousands of dollars
rather than the hundreds of millions required for Q.
Dr. Feng's team, which also includes Dr. Michael S. Warren and Eric H.
Weigle, began with a similar approach. But while traditional Beowulfs are
built from Pentium chips and other ordinary processors, Green Destiny uses
a special low-power variety intended for laptop computers.
A chip's computing power is ordinarily derived from complex circuits
packed with millions of invisibly tiny transistors. The simpler Transmeta
chips eliminate much of this energy-demanding hardware by performing
important functions using software instead instructions coded in the
chip's memory. Each chip is mounted along with other components on a small
chassis, called a blade. Stack the blades into a tower and you have a
Bladed Beowulf, in which the focus is on efficiency rather than raw
The method has its limitations. A computer's power depends not just on the
speed of its processors but on how fast they can cooperate with one
another. Linked by high-speed fiber-optical cable, Q's many subsections,
or nodes, exchange data at a rate as high as 6.3 gigabits a second. Green
Destiny's nodes are limited to 100-megabit Ethernet.
The tightly knit communication used by Q is crucial for the intense
computations involved in modeling nuclear tests. A weapons simulation
recently run on the Accelerated Strategic Computing Initiative's ASCI
White supercomputer at Lawrence Livermore National Laboratory in
California took four months of continuous calculating time the
equivalent of operating a high-end personal computer 24 hours a day for
more than 750 years.
Dr. Feng has looked into upgrading Green Destiny to gigabit Ethernet,
which seems destined to become the marketplace standard. But with current
technology that would require more energy consumption, erasing the
machine's primary advantage.
For now, a more direct competitor may be the traditional Beowulfs with
their clusters of higher-powered chips. Though they are cheaper and
faster, they consume more energy, take up more space, and are more prone
to failure. In the long run, Dr. Feng suggests, an efficient machine like
Green Destiny might actually perform longer chains of sustained
At some point, in any case, the current style of supercomputing is bound
to falter, succumbing to its own heat. Then, Dr. Feng hopes, something
like the Bladed Beowulfs may serve as "the foundation for the
supercomputer of 2010."
Meanwhile, the computational arms race shows no signs of slowing down.
Half of the computing floor at the Metropolis Center has been left empty
for expansion. And ground was broken this spring at Lawrence Livermore for
a new Terascale Simulation Facility. It is designed to hold two
More information about the Beowulf