[Beowulf] Building new cluster - estimate
matt at technoronin.com
Sat Jul 26 14:45:35 PDT 2008
On Wed, 23 Jul 2008, Ivan Oleynik wrote:
> In principle, we have some experience in building and managing clusters, but
> with 40 node systems it would make sense to get a good cluster integrator to
> do the job. Can people share their recent experiences and recommend reliable
> vendors to deal with?
I suggest that you need a minimum of 16GB/node (2GB/core) and possibly
You also should have a dedicated system for managing the cluster, a
commodity PC with a lot of disk is adequate. That is the ideal place to
host a DHCP server, tftp server (for for pxeboot), web/ftp/nfs server for
installation and local mirror of the various repositories. This system
should not be accessable by the users, just the admins.
You will want to set up IPMI on all of the nodes. You want to avoid
touching the harware or even going into the machine room whenever
Label both ends of each cable with source and destination. We have been
using LSL-79 labels from wiremarkersplus.com. I am very fond of using
velcro cable ties http://www.fastenation.com/class.php?id=9 for managing
cables. I personally bought a spool of the 8" straps for use on my
previous job and the folks at my current job bought a spool on my
recommendation. Very useful.
We have also been buying custom ethernet cables in 1' increments from
reynco.com. Very little price difference and ir radically reduced the
rat's nest of cabling. Color coding by function/network is handy as well.
Also, black ethernet cables inside of a black rack within a data center
that is really well lit can lead to all sorts of headaches. Release your
inner anti-goth and use brightly colored cables.
You will want to record the MAC address to physical location mapping
sooner rather than later.
Install blanking panels to cover unused spaces in the rack. This will
Both Rocks and Warewulf seem to be excellent and have very friendly and
supportive developers. Talking to them would probably be a good idea.
A much longer message than I planned, hopefully it is of use.
It's not what I know that counts.
It's what I can remember in time to use.
More information about the Beowulf