[Beowulf] some thoughts on thermal design, liquid cooling, etc.
James.P.Lux at jpl.nasa.gov
Mon Feb 14 09:17:00 PST 2005
It occurs to me that the real limiting factor in producing "cluster
oriented thermal design" is the volume of sales.
Say you want to design a custom motherboard/package for use in
clusters. This is, at a guess, probably a 3-5 million dollar project
(maybe down around a million if it's real close to an existing design).
Say the cost of a node is around a kilobuck or 2 (in plain, non-custom,
If you had a cluster with 1000 of those custom mobos, you're looking at
adding $3K/node to the cluster. That's a bit punitive... You could buy a
lot of machine room and cooling for that $3 mil.
Now, on the other hand, if you had 100 people willing to each buy a cluster
of this scale, then it's only adding $30-50/node, which is a lot more
Compare this to the consumer motherboard market (which, after all, is what
we are really using here...) A production run of several million mobos
isn't all that huge, so a Dell or HP can and do create customized
motherboard designs to meet some peculiar requirement (on-board
peripherals, etc.). Such customization only adds a buck to the mobo cost,
and presumably, that buck is made up in cheaper packaging, shorter cables,
one less manufacturing step, or somewhere.
Somehow, I doubt that the total sales of ALL motherboards for clusters, of
a given instance of motherboard design, exceeds a million units. Cluster
buyers tend to want different processors, different peripherals, etc., and
each configuration change would drive a whole new design cycle.
There is hope on the horizon. The increasing drive to "media computers" is
creating a demand for PCs that have high performance, but are quiet and
have good cooling. I have a Motorola Moxi BMC9012 "set top box" at home
from the cable company, and it is basically a Linux computer with an 80GB
drive and a some custom video hardware. It's also hideously noisy (for
something designed to sit in your living room) and dissipates >100W (all
the time.. there's no on-off button). There WILL be consumer pressure to
make it silent and to do better thermal management.
James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
More information about the Beowulf