1U P4 Systems

[John P. Quintana]

> hi william
>
> if you dont mind the "open frame design"...
> like at clusterCompute...
>
>	- keep in mind.. one can get a flat sheet of
>	metal and mount all the motherboard, disk, powersupply
>	onto a piece of sheetmetal .... to create a custom
>	very affordable/inexpensive 1U server rack
>
>	- it has better airflow due to the huge fans
>	in the cabinet....
>
>
>
>
> than you can easily mount 4 independent server into one 1U cabinet
> shelf .... 160 1U servers in one cabinet
> - <shameless plug> we can fit 3 independent server per
shelf</shamelessplug>
>
> and, you can also do that the kingstar way...http://www.KingStarUSA.com
> ( they are 2U servers... p3 slot-1 CPU vs the flat cpu ( socket370 )...
>
>have fun
>alvin
> http://www.Linux-1U.net -- 500Gb 1U Raid5 ...

I have been in touch with the guys at clustercompute.com and they are very
happy with their system.  In particular, I was worried about ground loops
since they are using metal rods to hold everything together.

We have been thinking about doing something similar.  If you look at our
current cluster http://www.dnd.aps.anl.gov/wulffnet/ you will see that we
have
had to put our cluster in a linear fashion across on a "big shelf".  This is
due to space restrictions and if we had to do it all over again, we
wouldn't.
The cabling issues get to be a real hassle since you are now dealing with
a lot of 50 foot CAT 5, KVM switch cable etc...  Our cluster is becoming
rather popular to the point that it is hard for us to shut it off or to
do development work since it is being used.  So, I am planning on building
another cluster (this time directly on our network so all Linux boxes
can send jobs to it via PBS.  Most of what we do is trivially parallyzable).
This would be for quick (i.e. 10 minute or less) type jobs and for me to
steal when
I need to do something :).

I think clustercompute really has a compact design for COTS hardware.
No sheet metal between motherboards means that you can pack more
boards together.  The location of the powersupplies makes sense and
what they really have is a 10 cpu module that can be replicated.  In
addition to being
custom, the kingstarusa.com boards still has a sheet metal rack between
motherboards
which limits their density.  In clustercomputes design they are limited by
the
headspace that they want above their cooling fan.

Being inspired by clustercompute.com, and also a WWW site that I ran across
a home based cluster a while ago (but can't find now) which showed the
motherboards in a
vertical configuration and held in place by plastic rails, we were going
to try and put together a system like http://www.clustercompute.com
but with rails holding the motherboards in a vertical position.  6" ATX
power extenders are available so that we can disconnect the motherboard
in place.  Rather than gluing floppies to the motherboard, we already
purchased
Linksys cards from http://www.disklessworkstations.com with etherboot ROMS
and
we already use the Wake-On-Lan feature in our current cluster to turn the
nodes
on.  (I also have a handfull of 4 MByte IDE Flash disks that could be used).
So... if we are lucky, we won't have to build the box with the buttons
and LEDS that clustercompute did (which was apparently a pain to build).  I
have been able to find ATX switches on the WWW for about $3.00 but there is
a packaging issue.  If I had to go this route, I might just buy a commercial
Digital I/O board and place it in a "master node" to turn everything on and
off
and also do the monitoring.  In some sense I agree with the arguments about
overpriced racks.  If we don't rackmount , we will probably use the Al
extrusions and panels
from http://www.8020.net. We have worked with their stuff in the past and
while it
isn't cheap, it does the job.  We might need to use PCI riser cards to pack
the ethernet
cards closer, but we will try it without first if we decide to go this
route.  I haven't
really seen this approach to much in building clusters so I was wondering if
there were any
good or bad comments people might have before we go buy a lot of stuff.

Cheers,

John

--
John P.G. Quintana             jpq at northwestern.edu
Northwestern University        Phone: 630-252-0221
DND-CAT                        FAX: 630-252-0226
Building 432/A008              http://www.dnd.aps.anl.gov
9700 South Cass Avenue
Argonne, IL  60439