[Beowulf] Clusters and Distro Lifespans

[Robert G. Brown]

On Tue, 18 Jul 2006, Gerald Davies wrote:

> John's comment about the short support lifetime of FC raises one of my
> concerns about distros and cluster set-ups in general.  In my
> department we have RH/FC based clusters.  When purchased they came
> with a pre-installed distro and have pxeboot/images.  I then spent
> time tuning them to our needs.
>
> My questions relating to this are:
>
> i)  Is the practice of buying clusters with pre-installed distros popular?

On the list, probably not -- many/most active list participants have if
anything been involved for years in developing just what goes into those
pre-installed distros to support a cluster.

Also, these days you almost certainly want to hook up ALL of your
systems, cluster nodes or not, to an update stream via e.g. yum.  Yum
permits automated update of libraries, kernel, and critical tools to
protect your system(s) against root exploits, provide bug fixes, and
sometimes to obtain functional updates in a timely way (especially if
you are using tools under active development that maybe aren't in a
standard distro".  It even facilitates YOUR distribution of locally
built RPMs to the cluster nodes in a timely and automated way (build
RPM, drop into cluster repo, next day at the outside all nodes have
update).

However, there are certainly folks out there for whom buying turnkey
clusters is appropriate.  If you are sysadmin poor and want a cluster
for a single, fairly specific purpose, then it makes lots of sense to
get a cluster from e.g. penguin on the one hand or a consultant like Joe
(on list) on the other hand and let them assume the responsibility of
providing update streams and managing most of your sysadmin and
management.

There is obviously an economic aspect to this.  For some cluster owners
it will be cheaper (by far) to effectively "outsource" this to a company
or consultant rather than hire a full-time cluster-trained sysadmin; in
other cases one is or has a sysadmin (or is very poor and cannot afford
the oursource and have to DIY even if you have to learn how along the
way:-) and doing your own distro etc. out of opportunity cost time is a
big win.

>
> ii) Would it be better to develop our own installation process for
> clusters so that upgrades, in terms of distros, can be rolled out
> easily?  I feel like i'm tied in some way to the supplier of our
> cluster for upgrades.

Yes.  In fact, I think that these days with FC or Centos you should
probably shoot for developing a kickstart/PXE installation from a
yum-supporting repo.  A "good" setup is one where to install or
reinstall requires that you reboot the system and wait a few minutes.

A second approach that is quite popular and that seems to work well is
to use warewulf to install a single exported cluster image and PXE boot
the cluster diskless.  Then maintain that cluster image with yum or
whatever.  The nice thing this is that you don't need cluster disks at
all, reducing node cost, removing a significant cause of node failure,
absolutely guaranteeing node OS homogeneity.

> iii) Do people regularly upgrade their clusters in relation to
> distros?  I guess this is like asking how long is a piece of string
> because everyone's needs are different.

I personally do and think it is a good idea.  With yum it is trivial to
do this, and yum also provides (as noted above) a very simple and
automated way to manage your LOCAL cluster software.  Pretty much
anything, including commercial tarball-distributed software, can be
rewrapped to install from RPM.  One of the things that I think has been
questionable in the past about some "cluster distros" or tools like
mosix has been the large gap between those distros and the primary
update stream of the "real" distro they were based on.  Updates are a
key component of maintaining security and functionality both, and yes, I
think security is important inside clusters (see previous post:-).

> Apologies if this sounds like a strange first post :)

Naaa, good questions.

    rgb

>
> Cheers,
>
> Gerald
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit 
> http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu