[Beowulf] Airflow

[matt jones]

Robert Fogt wrote:

 > Hello beowulf mailing list,
 >
 > My air conditioner will be enough for the amount of
 > heat
 > generated, but will I need circulation fans in the room also?

it depends on the design and layout of your cluster. mainly on the 
distance between the heat generating psu's and processors to the AC 
unit. the AC unit will only have fans powerfull enought to blow and suck 
air in a few meters, the rest on the way most AC units work is using the 
natural convection current that the hot and cooler air creates as it 
moves... it's the reasons why you have to have the windows closed to see 
the full effect of AC. using the advantages of hot air rises, means that 
for optimal use the AC unit is on the celing drawing in all the hot air, 
and pumping the cooled air out on the bottom, the floor. this would than 
rise the the nodes that would probably be best placed mid way.

this way you get a pool of cool air at the bottom, that the boards can 
draw from, and a hoter pool above that the AC can draw from and place 
cool air at the bottom. using all parts most efficiently. doing this i 
would say that if all boards were mounted in a single row of shelves 
like below, then you shouldn't need any fans other than th fans on the 
CPU and PSU, all the cooling would be done convectually.

 > I created a test cluster of 8 nodes, removing all the cases and mounting
 > everything onto a wire rack. It works fine with no heat problems. But
 > I am
 > concerned that once I build the entire 100 node cluster there will be
 > problems.

again depends how you design the whole cluster. particually how you 
stack the boards. in the horisontal,

|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |

would allow the heat to rise where as vertically stacked wouldn't allow 
this and the ones at the top would get the most heat.

also depends how many layers, or shelves you decide on using. i would 
personally only use one and have about 12/13 in a rows on the middle 2 
shelves, and use 4 racks in a square, have the AC unit in the middle of 
this square at the celing, alowing cool air to fall down the middle 
forming a coloumn of air, and a pool at the bottom. i would also mount 
all boards so that the I/O is facing outwards so that the components can 
get cool air from the coloumn.

i wouldn't put any more than two layers, because you might be lickly to 
get the top ones failing due to the heat of the others rising.

 > I was thinking an air purifier that is always on will slowly circulate
 > the
 > air when the air conditioner is off.

that would be a good idea, 'cos' when the processors are running less % 
they create less heat and your AC can turn off saving energy and money. 
i take it you have independant AC unit so you can have a thermostate 
attached without affecting anyone else. the key to using convection to 
cooling cluseters is keeping the circlation going, and the best way to 
do this is in an enclosed space, using a heat exchanger.

EMI is one thing you will have to watch... i have read that a 1mm steel 
case can reduce leakage by up to 75%, but i dont know how believeable 
that is or not. obviously hainf 100 nodes and 25 or more PSU's will 
create some amount of EMI and having nothing to absorbe this and the 
nodes being so close i would safely say that your lickly to have 
failings due to EMI. to prevent this i would place some steel sheets (if 
they absorbe EMI) between every other node.

a thought of mine is to have maybe 25 custome made or adatped cases, 
each having 4 boards and a single, shared PSU. if this were the case i 
would have one 120mm fan at the front and a second at the rear, both 
pushing air the same direction. i would also link these using air ducts 
to manifolds and then directly into the AC unit, providing it had a 
built in 'just fan' bypass. you could then take this to the extreem of 
management and have a single node monitoring the whole assembly, 
switching the AC on and off.. cases off and on as required, and even in 
line fans to route cold air to the cases that are producing the most 
heat. but that would require some enginearing to do effectivly and some 
programming of the 25 temp sensors (for each case), 25 power relays, the 
AC unit controls, fans and integrating with a job manager for the 
switching on and off of the nodes.

anyway, hope some of the above is usefull.
-- 
matt jones,

this e-mail has been virus scanned,
sent from a workstation called: matts, IP address: 10.64.63.17, Subnet
Mask: 255.255.192.1