[Beowulf] software for activating one of many programs but not the others?

[David Mathog]

On 2019-08-20 16:45, Lance Wilson wrote:
> There are quite a few packages that are very difficult to build on 
> Centos/RHEL ...

In my experience those fall into a couple of clear categories, most of 
which are best described as programmer sloppiness or indifference to 
portability:

1.  old code/bad code.  Programs used language features which were 
either never standardized or have been removed and either throw errors 
at build or explode when run.   Very high correlation with build methods 
which do not turn on  -Wall (or equivalent).   The only solution is to 
fix the code.  Common in programs nobody maintains anymore.

2.  cutting edge code.  The opposite of the preceding.  It was built on 
the very latest
Fedora or Ubuntu and uses features which only exist in those 
kernels/compilers/libraries.
Again, very high correlation with build methods which do not turn on 
-Wall (or equivalent).  In some instances using the latest devtoolset 
for Centos will help.  Other times a local copy of just one or two 
libraries are required (version 26 instead of the OS's version 23, that 
sort of thing.)  Otherwise a container is really the only option because 
they are essentially OS (version) specific code.

3.  hall of mirrors dependencies.  A needs B needs C,D which need E,F 
and G,H
etc. etc. etc. until one of the dependencies falls into categories (1) 
or (2) and so cannot be satisfied.  It is much more often (2) than (1).

4.  Anything using boost.  Boost can be very finicky about versions and 
build methods for programs which use it often are difficult to coerce 
into using a different version from an alternative location.  Also boost 
is notoriously hard to build on Centos, so if somebody has not already 
built a compatible version it may be time to move on.  Denis Arnaud is 
my hero in this regard, his boost RPMs have saved my butt numerous 
times.  His latest is 1.69 and boost is now at 1.71.  Are there features 
in 1.71 not compatible with 1.69 which somebody will use in code I need 
to build?  If history is any guide, absolutely.

5.  binaries only.  If a program is only available as a binary and it is 
dynamically linked then satisfying the library dependencies can be 
tricky.  Often this turns into
a special case of (2).

Issues which are rarely encountered and are not readily solved by 
containers:

5.  hardware dependency.  Not only was it developed/built on Alpha or 
Sparc, but it contains machine code for extra speed.  This used to be 
more common but since x86_64 hegemony it has largely gone away.

6.  other environment dependency.  This includes things like matlab code 
which has only ever been tested on Windows.  It is possible to make 
these run, sometimes, using the matlab run time library, but it is a 
PITA.  Using the matlab run time library isn't all that different than 
using a container - it is a huge block of code needed to run what is 
often a very small block of code.

Regards,

David Mathog
mathog at caltech.edu
Manager, Sequence Analysis Facility, Biology Division, Caltech