[Beowulf] clustering using off the shelf systems in a fish tank full of oil.

[Lux, Jim (337C)]

On 12/28/11 7:40 AM, "Prentice Bisbal" <prentice at ias.edu> wrote:

>There has been a company at the SC conferences for the past 3 years
>trying to sell exactly that (server cooling by submersion in mineral
>oil) for the past 3 years.
>
>In my opinion it, suffers from a few major problems:
>
>1. It's messy. If you every have to take hardware out of the oil to
>repair/replace, it's messy. The oil could drip all over, creating safety
>hazards. And if you need to remove a hardware component from a server,
>good luck! Now that everything is oily and slippery, there definitely
>will be a problem with that hard drive once it flies out of your hands,
>even if there wasn't a problem with it before!
>
>2. The weight of the mineral oil. Despite the density of current 1-U and
>blade systems, I still think that air makes up a not-significant
>percentage of volume of the full rack. Fill that space with a liquid
>like mineral oil, and I'm sure you double, triple, or maybe even
>quadruple the weight load on your datacenter's raised floor.
>
>

I've worked quite a lot with oil insulation in the high voltage world.
Prentice's comments (particularly #1) are spot on.

ALL oil filled equipment that is designed for servicing leaks.  ALL.
Maybe it's just a fine oil film on the outside, maybe it's a puddle on the
floor, but it all leaks.  (Exception.. Things that are welded closed with
oil inside, but that's not serviceable)

When you do remove the equipment from the tank, yes, it drips, and it's a
mess.   Slipperyness isn't as big a problem.. You lift the stuff out of
the tank, and let is sit for a long while while it drips back into the
tank.   Pick a real low viscosity oil (good for other reasons) and it's
not too bad. The problem is that there is some nook or cranny that retains
oil because of its orientation or capillary effects, and that oil comes
oozing/spilling out later.

Fluorinert is a different story (albeit hideously more expensive than
oil).  It's very low viscosity, has low capillary attraction, etc. and
will (if chosen properly) evaporate.  Equipment that cools by ebullient
(boiling) Fluorinert cleans up very nicely, because the boiling point is
chosen to be quite low.


I'm not sure I'd be plunging a disk drive into oil.  Most drive cases I've
seen have a vent plug.  Maybe the holes are small enough so that the oil
molecules don't make it through, but air does, but temperature cycling is
going to force oil into the case eventually.
>