[Beowulf] bring back 2012?

[Prentice Bisbal]

On 08/22/2016 11:46 AM, Stu Midgley wrote:
>
>
> On Mon, Aug 22, 2016 at 11:22 PM, Stu Midgley <sdm900 at gmail.com 
> <mailto:sdm900 at gmail.com>> wrote:
>
>
>
>
>         While the risk of an explosion is a certainly a theoretical
>         possibility, In practice, the risk of this is virtually
>         non-existent for a variety of reasons.
>
>         With water, the processors and other heat-generating
>         components would fail from the heat before the boiling point
>         of water is reached, so there would be little to no generation
>         of water vapor that could lead to an explosion. Also, any
>         heating/cooling system with water would be designed to
>         included an expansion tank to account for the thermal
>         expansion and contraction of water. There are millions, if not
>         billions, of homes and businesses in existence with hot water
>         heating systems, yet, I've never heard of any of them ever
>         exploding.
>
>         With Novec and other two-phase systems, the gas phase is
>         compressible, meaning it can store energy like a spring,
>         preventing or minimizing the case risk of an overpressure
>         situation rupturing the vessel. All that is required for this
>         to be used safely is an adequate volume for the gas, so that
>         is has excess 'capacity' to be compressed. This simple design
>         is what allows 20-pound propane tanks to be used all over
>         America (and probably other countries) to fuel gas grills and
>         be left out in direct sunlight all summer long, and be stored
>         directly under the heat-producing burners.  If those tanks
>         were filled to the top, they would explode in those
>         conditions, but but leaving about 1/3 of the tank empty, the
>         risk has been virtually eliminated. This was actually a top we
>         spent a lot of time discussing in my Chemical Engineering
>         Safety class in college.
>
>         This also applies to the tanks storing liquid nitrogen, liquid
>         oxygen, and other gases/liquid stored well below their boiling
>         point. Tanks of these substances can be found throughout the
>         world in industrial and laboratory environments, yet
>         explosions caused by them are quite rare. When they explode,
>         it's usually because someone who didn't know what they were
>         doing overfilled the tank, or the ambient temperature exceeded
>         the designed safety margins through some other catastrophic
>         event. (structure file, etc).
>
>         Finally, all systems where this is a risk would have plenty of
>         safety features to prevent this. My gas water heater at home
>         has a simple temperature/pressure switch to safely discharge
>         excess pressure/temperature event. These are cheap, readily
>         available items that you can buy at any local hardware store.
>         I also have a steam heat system in my house. In the early days
>         of steam heat, it was not unheard for a steam boiler to
>         explode with devastating results, but just to some simple
>         design elements (Hartford Loop) and basic mechanisms (low
>         water cut-off valve, pressure relief valves) have virtually
>         eliminated this risk.
>
>         Before I got in to HPC as a profession, I was a process
>         control systems engineer. My companies specialty was control
>         systems for boilers for power generation. The pressures of
>         these systems were much higher than what we're talking about
>         here. Our systems had plenty of pressure sensors, release
>         valves and failsafes. Incorporating any of these safety
>         elements into a cooling system like this is trivial, and I'm
>         sure the vendors who sell such solutions have already done
>         that where appropriate
>
>
>
>
>     But if you have 40kW of gear still running, your not storing the
>     liquid in the sealed container well below boiling point - its
>     actually the opposite you are running at or just above the boiling
>     point.  Even if you take the approach "our systems will shot down
>     if we loose the external cooling circuit)... that still takes time
>     to recognise and shutdown... mean while your systems are pumping
>     heat into the tank.
>
>     Again, with the boiler example, this isn't the sort of behaviour
>     you want in a computer room. You don't want this stuff venting...
>     and also, try and get a permit to operate such a system in an
>     existing or new facility.
>
>     With a non-phase change solution, this isn't an issue.
>
>
>
> FWIW the direct contact solutions (wether they use water or some other 
> dielectric fluid) as far as I can see have several main problems
>
>  * complexity (all that plumbing and getting it to 8 phi's + 2 cpu's 
> all crammed in 2RU)
This is why I said I'm glad someone else is doing all that engineering. ;)
>  * nodes have to be modified after the come out of the factory
I'm not sure, but I think the vendor (Dell, HP, etc.) sends the 
components to CoolIT or Asetek for final assembly. This sounds 
expensive, but car manufacturers have been doing it for years for low 
volume special additions where a 3rd party is more optimized for this 
small volume custom work than the large volume assembly lines. Yes, it's 
still a little more expensive, but not as bad as building something, and 
then taking it apart again to customize it.
>  * not all the components are cooled (ie. ram, disks etc)  You still 
> have to run some form of air cooling.
This is true. Most of the solutions I have seen have coolers for the 
CPUs, GPUs, and RAM, and leave the rest to ambient cooling, this covers 
the main heat producers, but some air cooling is still needed.

  But even with mineral oil, doesn't some of the heat leave the mineral 
oil and heat your data center, or is it cooled to a reasonable amount 
below ambient temperatures so it can absorb heat from the servers and 
still be at or below room temp?
>
> I've only run direct contact cooling on a desktop style box and that 
> was painful enough... let along on hundreds of servers in a rack etc.  
> I can't see how they will be price competitive, given all the 
> modifications that are needed to the systems (I've had pricing for a 
> single rack system but never purchased one).

Both CoolIT and Asetek claim to be only about 10%-15% more than 
air-cooling per rack, and that is quickly recovered through reduced 
cooling costs. I'm skeptical of the 10%-15% claim, but I have no doubt 
the cooling cost reductions would be significant.
> -- 
> Dr Stuart Midgley
> sdm900 at sdm900.com <mailto:sdm900 at sdm900.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20160822/58ca6719/attachment.html>