[Beowulf] CentOS/RHEL 7 & major upgrades

[Jörg Saßmannshausen]

Hi Olli-Pekka,

upgrading clusters is always a pain so I try to avoid it if possible. The guys 
here are planning to upgrade the College Cluster to the new RH version and 
they are running into all sorts of problems.

However, the flip side is: if you don't upgrade your are running into the risk 
that the OS is so old that newer programs cannot run any more, specially if 
you got a completely outdated glibc. Also, if you add newer hardware you want 
to make use of the new features (like avx, avx2 etc.) hence you will need to 
upgrade. 

I am using Debian for the clusters and that seems to work well. I done an 
upgrade of two clusters from squeeze to wheezy without major problems. I am 
currently installing jessie on the new hardware we got to replace some really 
old stuff. Here I took an image of the headnode and copied that over to a 
different disc. I then upgraded the old squeeze to wheezy and finally to jessie. 
I have not installed the compute nodes yet for other reasons. 

As for plan: I try to keep the OS reasonable new so new programs are running. 
I only keep the headnode up-to-date and the compute nodes are 'frozen', i.e, 
there will be no updates done. At least this way I hope to run a stable 
environment for production and the users can reproduce their work. 

The main problem I have with upgrading the OS is that I will need to re-
compile a number of programs. Most of the time that is working well but there 
are the odd one which take a week to re-compile due to various reasons. It 
usually takes a day or two. 

I hope that helps a bit.

It would be good to see what others are doing here as I am pondering: 
To upgrade or not to upgrade? That is the question!

All the best from a wet London

Jörg

On Friday 13 Feb 2015 08:00:52 Olli-Pekka Lehto wrote:
> Hi,
> 
> I’d be interested to hear if people have deployed or upgraded CentOS/EL 7
> on their clusters and what their experiences are? Also in the more general
> sense, what kind of distro upgrade philosophy do you follow? Are there a
> long-term roadmaps that you build internally, for example?
> 
> Recently I’ve had some internal discussions about the prospect schedule of
> upgrading our systems from CentOS 6.x to 7. I’m not really keen on
> upgrading a major distro version unless there is a _really_ compelling
> reason to do so, especially as we have driver dependencies to Mellanox IB,
> Lustre, Tesla and Xeon Phi as well as a lot of ISV codes. However, it
> would be nice to hear what the current pain/benefit ratio of such an
> upgrade is.
> 
> Best regards,
> Olli-Pekka
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf

-- 
*************************************************************
Dr. Jörg Saßmannshausen, MRSC
University College London
Department of Chemistry
Gordon Street
London
WC1H 0AJ 

email: j.sassmannshausen at ucl.ac.uk
web: http://sassy.formativ.net

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 230 bytes
Desc: This is a digitally signed message part.
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20150213/c50f5139/attachment.sig>