[Beowulf] [External] Re: immersion

[Prentice Bisbal]

On 3/24/24 2:38 PM, Michael DiDomenico wrote:
> i'm curious if others think DLC might hit a power limit sooner or 
> later, like Air cooling already has, given chips keep climbing in watts.

I've heard different numbers over the years, but water is something like 
3000x more effective as a cooling fluid than air. This is why water can 
be a much higher temperature going into the rack than air. A much 
smaller temperature difference is needed for the same heat transfer rate.

So processors can produce a lot more heat than they are now before we 
hit the same issue we're currently seeing with air. I don't think 
processors will ever get near that limit. I think other issues would 
keep them from getting there, like they stop working due to the high 
internal temps.

If we do start to hit limits with liquid cooling, either DLC or 
immersion, there's two knobs we can twist to help us: Increase flowrate 
and and decrease inlet temperature both of those will increase cooling 
capacity. Because a of the high thermal capacity and conductivity of 
water, a small temperature change can make a huge difference in heat 
transfer rates compared to air.

With air, we can't really do too much to practically increase flowrate. 
The flowrate through the chassis is dictated by two things: The 
resistance to flow through the chassis, and the pressure drop across the 
server. We can't do too much from a practical point of view to increase 
the pressure drop. We can't really create a significant vacuum in the 
hot aisle, nor can we create too much pressure on the inlet side. 
Neither situation really works in a data center that humans have to work 
in, and move into and out of regularly. And since air is compressible, 
as we increase the inlet pressure, it compresses and heats up... yadda, 
yadda, yadda...  Hot aisle and cold aisle containment and a few more 
fans is about the best we can do practically to reduce mixing and 
maximize airflow through the racks. Of course, as you add more fans, 
your PUE goes down.