[Beowulf] how useful is bproc, and what does Scyld cost?

[Sean Dilda]

There are some pros and cons to bproc (just like everything else).  My
experience with bproc is limited to some older versions of Scyld.  I've
never used clustermatic.

In my opinion, if you are already used the tricks to simplify
maintaining a decent sized number machines, then bproc hurts your more
than it helps you.  However, if you're not used to using things like
kickstart and yum to quickly add nodes and keep them up to date, then a
bproc system can make your life much easier.

There are definately some pitfalls to BProc.  As it was when I last used
it (late 2002), BProc systems read the binary off the master node, then
caused it to be run on the slave node (I'm simplifying things here)
without copying the binary onto the slave node.  This means there's not
really any filesystem you need to keep up on the slave node, which is a
big plus.  On the down side, there's no file system on the slave node. 
This means that if there are any files your program wants to access,
you're out of luck.  You have to know what they are beforehand and
either copy them over, or have them exported by NFS.

In your case, you're writing a custom app, so its possible that you can
work around the BProc limitations without much trouble.  But then again,
your problem may not allow that.  BProc's downfalls tend to manifest the
most with 3rd party apps.

As far as I know, Scyld and Clustermatic are the only distros with
BProc.  At one point Progeny was talking about doing some stuff with
BProc, but I don't know what ever came of that.

Another thing to note is that clustermatic and scyld probably don't have
the same version of bproc, beoboot, etc.  Some time back new versions of
the software was coming out of Los Alamos, but Scyld was reluctant to
use those newer versions, and tended to keep hacking on older versions. 
So Scyld may have some features clustermatic doesn't have, and vice
versa.

I don't think I can speak on how much Scyld costs.

If you're more interesting in having a turn-key cluster solution, I have
some friends who are happy with Rocks (http://www.rocksclusters.org/),
which is free.  However, I don't know how well that'll work with the
diskless setup you want.  As far as I know, Scyld/clustermatic are the
only ones that'll have a diskless setup "just work".  If that's what's
most important to you, then I say give them a try.  Otherwise, you may
want to look around.

Again I'll note, all my info is about a year and a half old.  Things may
(or may not) have changed since then.