[Beowulf] materials for air shroud?

[Prentice Bisbal]

Predicting airflow, especially in the turbulent regime, is not trivial,
and not always intuitive. Just ask John Hearns about that. He works for
McLaren's F1 team, and I bet they spend A LOT of computational power
studying the aerodynamics of their cars (I'm sure the exact amount is a
closely guarded secret).

Luxi, I'm sure, could pipe in, too.

>From my experience, that's one of the factors that sets the large server
companies, like Dell, HP, Sun, etc. apart from the smaller companies.
These larger companies have the larger resources and the  engineering
resources to model the airflow through their servers and it's effect on
heat transfer.

Here's an anectdote about this issue:

A certain national lab had incentives to use woman-owned and
minority-owned businesses. So when they needed to buy ~200 cluster
nodes, they went to a local small business that fit these criteria. I
never saw the actual servers myself but sounded like the company bought
generic 1U cases, and then put generic/commodity server components in
them. I'm sure they didn't to any airflow or thermal analysis on the
finished product.

After the cluster was brought online, they started having significant
hardware reliability problems, and overheating was one of those
problems, which could have been the cause of the other reliability
problems, since hardware failures usually increase with increased temps.

Moral of this story and all this noise? Airflow can be a tricky thing,
especially in a challenging conduit, like a 1U or 2U server.

--
Prentice



On 09/15/2011 06:33 PM, mathog wrote:
> A not so short story about air flow...
> 
> Yesterday I did some experimenting with different baffles and ducts, 
> each built temporarily
> out of the cardboard backs from yellow notepads and held together with 
> masking tape.  (Not worried
> about a fire, since it only ran for 10 minutes at a time like that, and 
> I was right there to
> yank the plug and rip out the cardboard if something went wrong.)  The 
> system has a Supermicro H8DC8
> motherboard in a Supermicro case.  This one:
> 
> http://www.supermicro.com/products/chassis/2U/823/SC823S-550LP.cfm
> 
> This is what that motherboard looks like without heat sinks:
> 
> http://www.supermicro.com/a_images/products/Aplus/MB/H8DC8_spec.jpg
> 
> and here is a very similar motherboard with heat sinks in place (but 
> not my
> motherboard, which uses conventional flat passive heat sinks, not the 
> big curved
> orange monsters in the picture).
> 
> http://i19.photobucket.com/albums/b165/TeamScream/Wide-2.jpg
> 
> Notice the 50% overlap in the heat sinks in the direction of the 
> airflow?
> That is the overlap with more conventional heat sinks too. Yes, it 
> really
> does feed hot air from CPU1 into CPU2.   Putting a little wall in,
> redirecting the hot air from CPU1 around CPU2 dropped CPU2's 
> temperature
> by 4C.  Nothing else I tried made a bit of difference - including 
> lowering
> the "ceiling" over CPU2.  CPU2 is still hotter than CPU1 even with that
> fix.  The reason it will not get any better is that while there are 4 
> fans
> in the system, they are not placed very well for this motherboard.  The
> first one sends all of its air into the PS and so doesn't cool the CPUs
> at all.  Totally a waste since the PS has a fan already.  The next fan
> is partially blocked by CPU1, so maybe 3/4 of its air is available for 
> CPU2.
> CPU1 then gets the remaining 1/4 of that fan, and most of the next fan,
> so around 1 whole fan's worth.  The last fan blows over the chipset and
> PCI slots, again, with no contribution to cooling the  CPUS.
> 
> For comparison, here is a Rio-works  HDAMA motherboard which we have.  
> For
> this design airflow was taken into account.  Note that the CPU sockets 
> are
> spaced farther apart perpendicular to the air flow.  It is very similar
> hardware otherwise:
> 
> http://www.opteronics.com/images/16a_MBLarge.jpg
> 
> there are some pictures of these with heat sinks in place which may be 
> found
> by google image search for "HDAMA motherboard" - I didn't want to link 
> to them as they
> all seem to be on ebay and those links could disappear at any time.  
> Note how the
> heatsinks do not overlap in the direction of the air flow?  We have one 
> of these,
> with passive heatsinks of approximately the same shape, but a bit 
> taller,
> stuffed into an old 2U case scavenged from an old machine.  In that 
> machine
> the two CPUs run at very close to the same temperature.  The component 
> layout
> in the case is very similar to the Supermicro except that the heat 
> sinks are not
> overlapping, so here there is a fan lined up directly on center with 
> each CPU,
> plus one to cool the chipset/PCI slots.  The PS gets by on its internal 
> fan.
> The old case has been "optimized" for air flow by the simple expedient 
> of placing
> the 3 fans as just described (originally there was just one fan in it), 
> plus
> removing the front panel and as much of the back panel as possible, 
> including
> the shield that normally goes around the jacks on the motherboard.
> 
> The HDAMA machine is pretty darn ugly, but it definitely "breathes" 
> better than
> the Supermicro.
> 
> I found a product with the perfect properties for sticking 
> polypropylene sheets
> together.  This is 3M "Jet-melt" 3731 hot melt adhesive.  (Also called 
> "Scotch-Weld").
> Unfortunately I need about 2cc of it, but nobody sells it in sizes less 
> than 11 pounds!
> The only place that sells anything in this whole 3M hot melt line as 
> single sticks is Digikey,
> and the one they sell
> 
> http://parts.digikey.com/1/parts/440266-hot-melt-adhesive-vo-5-8-x2-3748-vo-tc.html
> 
> is not as heat resistant as the 3731.  Probably have to use 3748 
> though, since at least
> it can be purchased easily.
> 
> Regards,
> 
> David Mathog
> mathog at caltech.edu
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>