[Beowulf] Building a new cluster - seeking some advice

[Joe Landman]

Jeffrey B. Layton wrote:

> Also, there are users in classified environments who would love to
> get rid of as many disks as they can because of administrative as well
> as security issues.

We had proposed something like that about a year ago to a customer with 
those issues.  Running nodes pure diskless, and doing the air-gap bit on 
the head node (simple power on/off was not acceptable, there was an 
explicit "air-gap" requirement on the power cord and wall socket).

It seems that some other vendor has productized this as well.  Generally 
it is fairly simple to do (Tiburon has support for this, as does 
Perceus/WW, and others ... ).

>> The diffentiation then is management.  I'm not convinced that it will be
>> easier to install and manage a cluster with (say) 1 to 4 GB flash drives
>> used as boot compared to using e.g. warewulf to manage boot images.  Or
>> that it will be faster.  Or (really) cheaper -- $40 is still $40 more
>> than a diskless system, and $40 that would buy it more real memory that
>> is likely to ultimately be more valuable in terms of improved
>> performance and stability.

Keeping the nodes as clean, simple, cheap as possible is a good thing. 
Cluttering them up with lots of unneeded things doesn't help much.

One area which does have a cross-over point is IPMI.  After about 10-15 
nodes, it starts getting less expensive to get a switched PDU and a 
console server.  Gives you the same bios level access and power control. 
  You do lose a little going without IPMI ... it does give you more 
scriptable management flexibility ...  one of the nicer aspects is being 
  able to force stateful installs (the disk based ones) to pxe boot on 
next boot, from a script.

IPMI cards cost $60-$120 depending upon what you need them to do (kvm 
over ip or not).  Dell has DRAC cards, Sun's are integrated in, HPs and 
IBMs are extra, no idea on pricing.  At 15 of thise, you are looking at 
$1800, which more than pays for the console server and switchable PDU.

> Good point. I agree :) Now, about teaching you Fortran90....

F90?  We are at F200x (x==3 or 4) already ... Sheesh!



-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615