[Beowulf] Gentoo in the HPC environment

[Jonathan Aquilina]

Ellis you are not the only one where distros such as CENTos have bit me in
the ass with their ancient versions of software, but ended up learning a
neat little perl hack so to speak.

When I tried gentoo a few years back it was a bit more involved then it is
now. you had to use 3 stage tarballs and now all you need is just the
stage 3 tarball to install. Another perk its like a rolling release so to
speak so you can ensure you are on the latest releases of software and its
easy to contribute and ensure the software is kept uptoday with filing a
bug only.

Could you clarify what you mean by weak module support?

Regards
Jonathan,

> Catching up:
>
> 1. My comment on loving Gentoo but not suggesting it for HPC deployment
> assumed old-style installation across the entire cluster as the sole OS.
>   Obviously netboot/VM/Docker beats the hell out of such a static
> implementation, I just wasn't interpreting the OP's question that way,
> which was my bad.  Joe nailed it in this regard.
>
> 2. My new place of work (Panasas) supports RHEL 6 (maybe 7, I'm green
> here), so it's there's one datapoint where RHEL is still supported.  We
> also support a bunch of other distros though, so it's certainly not
> exclusive.  I /hope/ to someday push for Gentoo weak-module support.
> We'll see about that.
>
> 3. I cannot count on my hands and toes the number of times restrictive
> distros like RHEL bit me in the ass when I tried to run any number of
> HPC applications to push my new/shiny/academic file system.  Holy smokes
> what I would have given for a flexible environment then.
>
> 3.a continues below:
>
> On 06/25/2014 04:50 PM, Prentice Bisbal wrote:
>> On 06/25/2014 03:08 PM, Kilian Cavalotti wrote:
>>> On Wed, Jun 25, 2014 at 10:29 AM, Andrew M.A. Cater
>>> <amacater at galactic.demon.co.uk> wrote:
>>>> RHEL doesn't cut it for these people: they know that they want later
>>>> GCC / different commercial compilers / hand written assembly -  a
>>>> later
>>>> kernel with a smarter scheduler ...
>>>>
>>>> SCL really doesn't work - it's stil not up to it.
>>> One way to deal with this is to separate user applications from the
>>> OS, as much as possible. And compilers could be considered as user
>>> applications.
>>> You can just use a very minimal OS on your compute nodes, then compile
>>> and install all the user facing bits in a shared location. You hand an
>>> environment modules system to the users and off they go. Systems such
>>> as EasyBuild (https://hpcugent.github.io/easybuild/) aim to facilitate
>>> this by allowing easy compilation and installation of scientific
>>> software (based on descriptive specification files, Ã  la Gentoo
>>> ebuilds), including dependencies, and by automatically generating
>>> environment modules.
>>>
>>> This way, you don't really care what the underlying OS is. You can
>>> have as many versions of GCC, Python, R, Perl, Ruby or anything
>>> installed alongside each other with no side effect, as long as you
>>> load the right module before running your job. It's like a
>>> distro-agnostic ebuild system.
>>>
>>> You can keep the distro the hardware vendor recommends to retain
>>> support (for interconnect drivers, parallel filesystems and such)
>>> while making your users happy with the newest versions of the software
>>> they need^Wwant.
>>>
>> I agree with this approach. I've been doing this for years, and it's
>> really not has hard as people make it out to be. There's the occasional
>> 'dependency Hell' situation, but that's not usually that bad unless you
>> are building a GUI application. Fortunately, GUI users aren't too common
>> in HPC. Overall, I find compiling Perl, R, etc. from source and
>> installing each version in it's own installation directory much easier
>> then learning how to get package managers to allow you to install
>> different versions of the same packages in a sane way.
>
> 3.a: Agree with some of the above from Kilian and Prentice.  The
> absolute best scenario (IMHO) for users and administrators is to provide
> an environment where users who want full control and want to take the
> responsibility for support onto themselves (e.g., me and all my fellow
> systems researchers in my PhD) can do so, while users who want all of
> that "nonsense" to get out of the way so they can run their application
> X to do "real" science without having to recode it for bleeding-edge (or
> super-old) version of GCC/Perl/Python/etc can also do so.  The ONLY
> place this is possible is in a flexible (VM/Docker/etc) environment.
> Administrators just need to tell the latter group "choose from ImageA,
> ImageB, and ImageC to run your application."  Counter-argument will be
> "I have to manage those images."  Counter-counter argument is "Is that
> really harder than rolling special environments to cope with those users
> (which you'll have to do anyhow)?"
>
> I ended up doing very crazy root-stealing, chroot-establishing things to
> get my science done in my PhD.  If you prevent intelligent people from
> doing their work, they are going to be your worst nightmare.  Don't kid
> yourselves if you think you are doing anyone favors by providing
> super-static OS environments like RHEL for your users.  You are just
> being lazy (and not the good kind of programmer lazy).
>
> Best,
>
> ellis
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>