[Beowulf] More about those underwater data centers

[Lux, Jim (337K)]

I suppose, we are talking about HPC here with already large infrastructure, so the “incremental pain” from having to deal with immersion cooling is probably smaller than the “incremental pain” from having to air cool a substantially larger facility – floor space costs money in many ways.  Having a rig to hoist electronics out of the vat (or drain the tank, or however) isn’t all that much different than having any other specialized support equipment –

If you had a few dozen processors, probably not worth it – get up to 1000 nodes, and it winds up being small in comparison.

However, I think it is still pretty exotic – you guys have it figured out, and that’s probably part of your secret sauce – I wouldn’t recommend it for the “casual” or “small” HPC installation.

OTOH, if someone wants to try it, have at it – it does work really well from a thermal standpoint – moving liquid around is MUCH more efficient than moving air around.



Jim Lux
(818)354-2075 (office)
(818)395-2714 (cell)

From: Stu Midgley [mailto:sdm900 at gmail.com]
Sent: Monday, November 05, 2018 3:31 PM
To: Lux, Jim (337K) <James.P.Lux at jpl.nasa.gov>
Cc: Jonathan Aquilina <jaquilina at eagleeyet.net>; ghenriks at gmail.com; Beowulf List <beowulf at beowulf.org>
Subject: Re: [Beowulf] More about those underwater data centers

I refute both these claims.

You DO want to run your boards immersed in coolant.  It works wonderfully well, is easy to live with, servicing is easy... and saves you almost 1/2 your power bill.

People are scared of immersion cooling, but it isn't that difficult to live with.  Some things are harder but other things are way easier.  In total, it balances out.

Also, given the greater reliability of components you get, you do less servicing.

If you haven't lived with it, you really have no idea what you are missing.


Serviceability is NOT challenging.



You really do NOT want to run boards immersed in coolant - yeah, there's folks doing it at HPC scale

Whatever the coolant, it leaks, it oozes, it gets places you don't want it to go. And serviceability is challenging. You need to pull the "wet" boards out, or you need to connect and disconnect fluid connectors, etc.  If you're in an environment where you can manage that (or are forced into it by necessity), then you can do it.
--
Dr Stuart Midgley
sdm900 at gmail.com<mailto:sdm900 at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20181106/24753f28/attachment-0001.html>