[Beowulf] distributions

[Robert G. Brown]

On Fri, 27 Jan 2006, Greg M. Kurtzer wrote:

> On Thu, Jan 26, 2006 at 04:09:58PM -0700, Warren Turkal wrote:
>> Is it ok to mix linux distributions when building a cluster? I am wondering
>> for migration purposes. For instance, it the current cluster had FC2 and I
>> wanted to move to FC3, would it be okay to install new nodes as FC3 and
>> gradually migrate?
>
> One of the design goals of Fedora is to NOT be a production capable OS.
> This is done by limiting its life to only 6 months with no guaranteed
> smooth update path (among several other things). With that said....
>
>   Why maintain this OS on a cluster or any system for that matter that
>   requires stability?
>
> You are not the only person doing this by far, yet it is something that
> I just never understood. I would love to hear some reasoning as to why
> people use Fedora in production where there are so many other distros
> that are actually designed for production use (and are free).

FC isn't all that bad.  In fact, it is pretty good (he says, typing on
it).  For a desktop environment, having a rapid development cycle is
something of an advantage, as here it pays to closely track e.g.
kernels, libraries, device drivers and the newer cooler applications.
For a cluster node or cluster server, for MOST purposes the operating
system/distro flavor, variant, age are irrelevant as long as they
install easily, run moderately stably, and have a transparent update
mechanism a la yum (or apt).  Here, too, having a rapid cycle is a
tradeoff -- it's more likely to find the latest NICs and libraries (e.g.
GSL, for one important example) at the expense of having to mess with
things a bit to work out a stable configuration when a new release comes
out (something that we've found you have to do anyway with ANY distro,
no matter how "stable").

What we do is use centos for servers (LAN/department servers, that also
serve the cluster nodes with e.g. home and project space).  We use
FC-even revision numbers for desktops, cluster nodes, etc.  We skip odd
FC releases to stretch the upgrade cycle out to a year.  This worked
pretty well during the i386->x64 transition a couple of years ago,
except that we had to hop from FC1 to FC2 at that time ON the six month
boundary to get "proper" x64 support.  We ARE also starting to use
centos for nodes -- no reason not to, and no real religious or practical
reason to use FCx.  It's more a question of whether FCx (at any given
moment) has something you need relative to Centos.  Right after a Centos
release that is unlikely to be true -- Centos 4 "is" FC4, for all
practical purposes.  When FC6 comes out, though, there may well be a
good reason in there to convert to FC6.

Note well that for most folks, once you get a cluster node installed and
working, there is little incentive to EVER upgrade the OS except for
administrative convenience.  We still have (I'm ashamed to say) nodes
running RH 7.3.  Why not?  They are protected behind a firewall, they
are stable (obviously:-), they work.  They are also slated for removal
-- at the end of their hardware life cycle -- and it isn't even worth
the time required to flash them forward to FC4 or Centos 4 to freeze
again.  One should really make the decision about what distro to use on
the basis of what they need, BALANCING e.g. device drivers, library
advances, application advances against the administrative hassle of
reinstalling.  Nowadays with PXE/kickstart etc. the latter isn't really
so great for most node hardware -- pretty low boundary to surmount.  For
the USERS an upgrade or changeover is usually invisible anyway -- at
most their apps will need a rebuild if their binaries can't cope with
the new shared libraries.

    rgb

>
> Thanks. :)
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu