[Beowulf] Gentoo in the HPC environment

[John Hanks]

It seems to me that this question is not unlike suggesting what size tires
someone should buy without first asking what kind of car, truck, motorcycle
or bicycle they use and what they use it for. In my environment many of my
users struggle just using the command line to submit jobs and as I divide
my time it is pretty clear that spending the majority of it helping users
get jobs running on a relatively stable, if out of date and non-optimal,
CentOS 6.x system is much more beneficial to the organization than spending
than time trying to get a few percent more performance out of the kernel
and libraries. If I could get that performance for "free" as a side effect
of having installed a stable distribution that took very little of my time
to manage, that would be great. But that level of performance will never be
a priority over just educating and getting users on the system.

Would I rather be working on setting up an optimized cluster using CoreOS
or Gentoo and putting applications in Docker containers? Yes, yes I would.
Would spending time doing that provide any benefit to my current userbase?
Not in the slightest and would actually be quite negative to overall
productivity. I don't think which distribution is the correct question to
ask until you've clearly addressed exactly what it is your users are trying
to accomplish and the distribution (or level of optimization) is selected
to meet that need. What are they driving, where are they driving to and
what are they hauling; then buy tires.

jbh



On Sat, Jun 28, 2014 at 7:38 AM, Jonathan Aquilina <jaquilina at eagleeyet.net>
wrote:

> Correct me if I am wrong here, and apologies for the delay as this thread
> and discussion has really taken off.
>
> My arguments for gentoo is more along the lines of optimization of the
> source code for ones hardware.
>
> You mention CentOS and Red hat how optimized is the heart of the system
> for the hard ware its running on? How secure are they as an os? Reason I
> bring up these questions is that I feel that resource are wasted that
> could be used for more important things.
>
> Another thing that I like about gentoo is the hardened profiles as well as
> the extra security features that one can setup at the kernel level as
> well, but that is going off my point.
>
> I for instance currently even though nothing is running on it have a vps
> with 2gb of ram and with no websites or db on there yet just the servers
> ram wise i use about 62mb. I look at the low memory foot print as an
> advantage in the sense that the rest of the ram can be used for more
> important things.
>
> > Hi, Jonathan.
> >
> > I would make a strong argument against Gentoo.  I would recommend that
> > you choose Red Hat or its binary-compatible derivatives, like CentOS.
> >
> > At the end of the day, you need to ask yourself what you are trying to
> > accomplish.  In an HPC environment, that should be a customer-facing
> > service that is reliable and well supported.  You do not want to be
> > constantly patching, compiling or changing APIs out from underneath of
> > research code.  Enterprise Linux provides a stable base for HPC
> > application development with a very solid lifecycle.
> > https://access.redhat.com/site/support/policy/updates/errata/
> >
> > Being part of a larger community, running the same builds, has its
> > advantages.  You won't be the only person encountering a weird stability
> > or performance bug.  You also get vendor hardware support, which is
> > huge.
> >
> > I know the counter arguments here.  There is always going to be the
> > coder that wants Ubuntu and this-month's release of $LANGUAGE, like on
> > their vagrant box.  I have found the Software Collections to be a
> > fantastic solution to this demand.  You get the stability of an
> > enterprise OS, plus the latest language features for developers.  This,
> > along with environment modules, is win-win.
> > http://wiki.centos.org/AdditionalResources/Repositories/SCL
> >
> > Even if you are a one-man shop, ask yourself what you want to be doing.
> > Do you want to be a systems programmer, or do you want to be a
> > research application developer.  Your time is precious.  Choose wisely.
> >
> > Cheers.
> >
> >
> > On Tue 06/24/14 08:43AM +0200, Jonathan Aquilina wrote:
> >> Hey guys,
> >>
> >> I am just curious as I have recently switched to Gentoo for every day
> >> use
> >> due to its customizability and how everything is compiled for a
> >> particular
> >> system.  Has anyone used Gentoo in an HPC application?
> >>
> >> Regards,
> >> Jonathan
> >>
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
> >> To change your subscription (digest mode or unsubscribe) visit
> >> http://www.beowulf.org/mailman/listinfo/beowulf
> >
> > --
> > Gavin W. Burris
> > Senior Project Leader for Research Computing
> > The Wharton School
> > University of Pennsylvania
> >
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140628/8bcca4a1/attachment-0001.html>