[Beowulf] software for activating one of many programs but not the others?

[Skylar Thompson]

We also use Environment Modules, with a well-established hierarchy for
software installs
(software-name/software-version/OS/OS-version/architecture). Combined with
some custom Tcl functions and common header files for our module files,
this lets us keep the size of most module files very small (2-5 lines).

If we were to do it again today, maybe we'd use Lmod, but Modules is
functional and has a lot of inertia.

On Tue, Aug 20, 2019 at 06:50:31PM +0000, Ryan Novosielski wrote:
> Really sounds like you should be using environment modules. What I’d recommend to anyone starting out today would be Lmod: https://lmod.readthedocs.io/en/latest/
> 
> Most of the software building/installation packages interface with it.
> 
> Generally the software installs are done into a place that’s unique for each package and version, and maybe even for what compiler it was built with (see hierarchical).
> 
> --
> ____
> || \\UTGERS,  	 |---------------------------*O*---------------------------
> ||_// the State	 |         Ryan Novosielski - novosirj at rutgers.edu
> || \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
> ||  \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
>      `'
> 
> > On Aug 20, 2019, at 1:11 PM, David Mathog <mathog at caltech.edu> wrote:
> > 
> > On a system I am setting up there are a very large number of different software packages available.  The sources live in /usr/local/src and a small number of the most commonly used ones are installed in /usr/local/bin, /usr/local/lib and so forth.  The issue is that any of the target end users will only want a couple of these.  If they were all fully installed into /usr/local there would be some name conflicts.  They may also be bringing some of their own versions of these, and while $PATH order can help there, it would be best to avoid those possible conflicts too.  Users don't have priv's to modify /usr/local, so they cannot install/uninstall there themselves.
> > 
> > So I'm looking for something like
> > 
> >  setup software_name install
> >  setup software_name remove
> > 
> > which would install/uninstall the packages (perhaps by symlinks) from
> > 
> >  /usr/local/src/software_name
> > 
> > under the user's home directory.  The goal is that the setup scripts NOT be constructed by hand.  It would have a
> > 
> >  setup software_name install
> > 
> > which would emulate a:
> > 
> >  make install
> > 
> > and automatically translate it into the appropriate setup commands.  Some of these packages have hundreds of programs, so anything manual is going to be very
> > painful.
> > 
> > Anybody seen a piece of software like this?
> > 
> > I don't expect this to work in all cases.  Some of these packages hard code paths into the binaries and/or scripts.  The only hope for them is for the user to do some variant of:
> > 
> >    cd $HOMEDIR
> >    (cd /usr/local/src; tar -cf - software_name) | tar -xf -
> >    cd software_name
> >    make clean  #pray that it gets everything!!!
> >    ./configure --prefix=$HOMEDIR
> >    make
> >    make install
> > 
> > There is a file which documents how to build each package, although it is nowhere near complete at this time.
> > 
> > Docker is already available if the user wants to go that route, which avoids this whole issue, but at the cost of moving big images around.
> > 
> > Thanks,
> > 
> > David Mathog
> > mathog at caltech.edu
> > Manager, Sequence Analysis Facility, Biology Division, Caltech
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> > To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit https://beowulf.org/cgi-bin/mailman/listinfo/beowulf

-- 
Skylar