[Beowulf] Gentoo in the HPC environment

[Prentice Bisbal]

On 06/25/2014 03:08 PM, Kilian Cavalotti wrote:
> On Wed, Jun 25, 2014 at 10:29 AM, Andrew M.A. Cater
> <amacater at galactic.demon.co.uk> wrote:
>> RHEL doesn't cut it for these people: they know that they want later
>> GCC / different commercial compilers / hand written assembly -  a later
>> kernel with a smarter scheduler ...
>>
>> SCL really doesn't work - it's stil not up to it.
> One way to deal with this is to separate user applications from the
> OS, as much as possible. And compilers could be considered as user
> applications.
> You can just use a very minimal OS on your compute nodes, then compile
> and install all the user facing bits in a shared location. You hand an
> environment modules system to the users and off they go. Systems such
> as EasyBuild (https://hpcugent.github.io/easybuild/) aim to facilitate
> this by allowing easy compilation and installation of scientific
> software (based on descriptive specification files, à la Gentoo
> ebuilds), including dependencies, and by automatically generating
> environment modules.
>
> This way, you don't really care what the underlying OS is. You can
> have as many versions of GCC, Python, R, Perl, Ruby or anything
> installed alongside each other with no side effect, as long as you
> load the right module before running your job. It's like a
> distro-agnostic ebuild system.
>
> You can keep the distro the hardware vendor recommends to retain
> support (for interconnect drivers, parallel filesystems and such)
> while making your users happy with the newest versions of the software
> they need^Wwant.
>
I agree with this approach. I've been doing this for years, and it's 
really not has hard as people make it out to be. There's the occasional 
'dependency Hell' situation, but that's not usually that bad unless you 
are building a GUI application. Fortunately, GUI users aren't too common 
in HPC. Overall, I find compiling Perl, R, etc. from source and 
installing each version in it's own installation directory much easier 
then learning how to get package managers to allow you to install 
different versions of the same packages in a sane way.

Prentice