[Beowulf] Innovative liquid cooling

Fri Mar 1 15:46:23 PST 2013

Hi all,

answers are inserted.

On Donnerstag 28 Februar 2013 Prentice Bisbal wrote:
> On 02/28/2013 05:00 AM, Hearns, John wrote:
> > I think this has been discussed here before, but it is a pretty
> > innovative product:
> > 
> > http://www.theregister.co.uk/2013/02/28/wet_servers_cut_cooling_costs_res
> > earch_leeds_university/
> 
> I really should just save my rants about immersive liquid cooling on a
> web page somewhere so I can just provide a link every time this topic
> comes up. I'd just provide links to my previous rants here in the
> archives, but I'm feeling lazy this morning.
> 
> Seriously, this article is both interesting and confusing, and there are
> some stupid/ridiculous statements in the article.
> 
> 1. The interesting:
> 
> How easy is it to clean this liquid up? Is it oily like mineral oil? If
> it's not slippery and it's easy to clean up, that addresses my biggest
> problem with immersive cooling. The Novec didn't look as thick or
> slippery as mineral oil in the video, but it's hard to tell something
> like that from a web-quality video.

It evaporates easily and it is used as a cleaning agent. 
You can dip your mobile into it and it is still working (the mobile, that is). 
When you fish it out again you got a clean mobile. You might be surprised how 
much grease there is on a mobile ;-)
So it is not like oil, the visosity is lower.

> 
> 2. The confusing:
> 
> Is the Icetope system an immersive cooling system? From the video, it
> looked more like a "direct contact" cooling system where the liquid is
> run through pipes into "direct" contact with the processors, but the
> demo made me think they are talking about immersive cooling. Even for
> direct-contact cooling (if that is the right term) having a
> non-conductive liquid is a better option than water if a leak occurs.

Yes. Motherboard is imersed into the Novac in a sealed Aluminium container 
which got a chilled plate on one side. The chilled plate is cooled with water. 
So for that you need a pump. The excess heat is removed by an outer loop water 
cycle. Here you can use whatever you like, even grey water (with a filter). We 
want to use 3 radiators so here we need a heat-pump, the same you get from 
your local DIY store for your domestic central heating. Not much noise here.

> 3. The stupid/ridiculous:
> > Dunking servers in new magic liquid 3M Novec reduces the cooling
> > system's energy use by 80 - 97 per cent, compared to cooling systems
> > that use air. Air cooling is inefficient because it is a poor
> > conductor, produces diffuse general heat and requires energy-chomping
> > high powered fans, said the boffins. 
> > 3M Novec is also a thousand times better at conducting heat than
> > water, and one low-powered fan in a chamber of 3M Novec is adequate to
> > chill a server array.
> 
> At some point or system size, you're still going to need pumps to
> circulate the liquid. While natural convection is fine for gases, it's
> usually inadequate for liquids due to their higher viscosity. And if you
> need to transport that liquid horizontally away from the heat source to
> the heat sinks, you're definitely going to need pumps. Liquids might
> have thermal capacities and thermal conductivities that are about 1000x
> that of air, but I think the viscosity of a liquid has got to be at
> least 1000x that of a gas. In this case, the pumps are still probably
> using less electricity than all those fans, but I think these quotes
> distort some of the facts.

See above. The novac will not be pumped around, convection is all you need. 
Remember, the nodes are standing upright and are not flat as a normal 
installation would suggest. So the transport is vertically and here you got a 
'chimney effect' as well. 
So, as mentioned above, you need two sets of pumps for the inner and outer 
look of the water (sic!) cooling system. 

> 
> > The fact that this system is completely enclosed raises a host of
> > possibilities. It does not interact with its environment in the way an
> > air-cooled server does, so you could put it in an extreme environment
> > like the desert
> 
> Ummm, no. that heat still has to go somewhere. And that "somewhere" has
> to be at a temperature low enough for there to be a temperature
> difference large enough create the "driving force" necessary for useful
> heat transfer. That's probably not going to happen in the desert.

The outer water cycle has a max. temperature of 50 °C and the return is max 45 
°C. Given it is cold in a dessert at night you might be able to just about do 
that. However, I would not really want to have my cluster in the dessert ;-)

> > It is also completely silent. You could have it on a submarine or in a
> > classroom.
> 
> See my earlier comment about the need for a pump. I guarantee that even
> the smallest production systems will need some kind of circulation pump.
> you can probably locate that pump further away from the system being
> cooled, but it will be producing some level of noise, somewhere.

Think about how noisy a domestic heat pump for your central heating is and you 
get an idea how loud the cluster is. I stood next to one and you could barely 
hear it. So no big pumps.

> Overall, if this 3M Novec overcomes the drawbacks of mineral oil, this
> is great, but I feel that this article and this research is more like
> press release for 3M Novec.

Well, 3M is supporting the project. However, to be honest, of all the systems 
I have seen so far that seems to be the most intelligent one, specially as you 
can harvest the heat and do something with it. 
The radiators we are using are too small to heat up the building or the 
staircase in the winter, granted. However, it is more to demonstrate the 
principle and I only get one rack. I will post my experiences when and if I 
get the system installed. At least so far nobody here mentioned a serious 
problem which is reassuring.

> Rant over. You may now return to your regularly scheduled work day.

Not a rant, just some comments :-)

All the best from a dark London

Jörg 

-- 
*************************************************************
Jörg Saßmannshausen
University College London
Department of Chemistry
Gordon Street
London
WC1H 0AJ 

email: j.sassmannshausen at ucl.ac.uk
web: http://sassy.formativ.net

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html