[Beowulf] A press release

[Joe Landman]

Prentice Bisbal wrote:

> Here's another reason to use tarballs: I have /usr/local shared to all

eeek!!  something named local is shared???

FWIW: we do the same thing, but put everything into /apps, and all nodes 
have mounted /apps ...

requires a little ./configure -prefix=/apps/...

magic, but it works well.

> my systems with with NFS. If want to install the lastest version of
> firefox, you can just do this:
> 
> cd /usr/local
> 
> tar zxvf firefox-x.xxx.tar.gz
> 
> cd /usr/local/bin
> 
> ln -s ../firefox-x.xxx/firefox .
> 
> Now all users can use the latest version of firefox (/usr/local/bin is
> in their path, and comes before /usr/bin, usr/X11R6/bin, etc.)

Oddly enough, I am not a huge fan of dumping lots of binaries into one 
path.  Part of the reason is the package management one ... all you need 
is one renegade package and a packager that things [s]he is smart, and ...

> With RPM, deb, or whatever, I'd have to use func or ssh and a shell
> script w/ a loop to install on all systems (assuming nightly 'yum
> update' cron job won't work in this case)

If you don't know about pdsh ... it will increase your karma.  We 
install it on every cluster we build.  Makes life soooo much easier.

> This is incredibly helpful with Python, Perl, R, and other languages
> which have additional modules or libraries. Installing additional
> modules can be very easy (CPAN module for perl, for example). These
> modules aren't included in RPM format (that I know of), and when you
> upgrade, perl or python, the RPMs clobber whatever modules you installed
> in /usr.

Yup.  Bioperl is a great example.  Of course to install that shared, you 
need perl/modules installed shared.

> 
> Compiling Perl can be time consuming vs. just installing the RPM, but
> once installed, if I run '/usr/local/bin/perl -MCPAN -e shell' as root,
> I can install all the perl modules needed just once, and they won't be
> clobbered by an RPM update. In the end, this is much more efficient. And
> CPAN manages dependencies automatically, too.

We usually build our own Perl these days.  Having had some interesting 
... experiences ... with vendor compiled versions, we decided to forgo 
their "assistance" and do it ourselves.  Seems to work much better.  And 
we have the process (including the module builds) automated quite nicely 
now.  We use it for SICE ... er ... DragonFly.

> 
> We use RT where I work, which requires a few Perl modules. On Friday,
> June 13 (Yes, Friday the 13! - it figures. ). I had to stop and restart
> our web server that provides RT. The perl packages had recently been
> updated. When apache restarted, it couldn't find the necessary perl
> modules, and RT wouldn't function. It took me HOURS to track the problem
> down to a couple missing perl modules.

Yup.  This is why we use a different tree than the vendor supplied ones. 
  We can tie back into the vendor supplied web server .... or use our 
own (usually do the latter).  Upgrades can be a crap shoot, even on the 
best of intentioned systems.  Right now we have run head first into a 
Ubuntu problem with OpenVPN on a system, where we upgraded the server 
after the OpenSSL fiasco, and suddenly CRLs no longer worked.  Fixed a 
config file by hand.  Next upgrade?  Same bug.  Sigh.

Package management is good ... for ... um ... er ... what was that 
again? Making sure your stuff doesn't break when you update/upgrade? 
Oh.  Maybe they will get around to making sure that actually is the 
case?  FWIW:  we have seen the *same* sorts of problem with RPM, apt, 
yum, suse's monstrosities (zmd and others), ...  they are all broken in 
subtle ways that most folks don't run into.  Its only when you have a 
system you needed to make specific changes to, that these changes get 
lost on the next "up"grade.  We need more of a change management system. 
  Mercurial for systems config.

Grrr....

Joe


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615