[Beowulf] Gentoo in the HPC environment

[Andrew M.A. Cater]

On Wed, Jun 25, 2014 at 11:30:52AM -0400, Joe Landman wrote:
> On 06/25/2014 09:51 AM, Gavin W. Burris wrote:
> >Hi, Jonathan.
> >
> 
> Unfortunately, the reality of the HPC code market is that, quite
> often, the OS required by the application for support is often at
> odds with what you describe above.  More often than not, commercial
> and closed source applications are built and qualified (for support
> and guarantee of functionality) against several very specific OS and
> library versions.  It is rare, in my experience with this, that any
> of these are up-to-date versions of Red Hat or Red Hat derived
> distributions.
> 

Amen, brother :) I think the more interesting case is where you have
a community of smart people wanting to use code they've written themselves
- maths-y / science-y / high performance programmers / genomics types or the like who
aren't running prepackaged code or simulations but can stress your
cluster to the utmost.

RHEL doesn't cut it for these people: they know that they want later
GCC / different commercial compilers / hand written assembly -  a later
kernel with a smarter scheduler ...

SCL really doesn't work - it's stil not up to it.

Then there's a creative tension with $large_hardware_cluster_supplier
who supplies Red Hat Enterprise Linux at a fixed point version / has 
hardware specific drivers and you'll have to support this for the 
foreseeable future. Red Hat won't support your cluster machines unless
you pay them extra per node to stay pinned to a specific version for a couple 
of years. 

At this point, you might as well get your local team to support Debian
on this - and you will have all the extra packages that Debian may
provide over RHEL :)

Scientific - from CERN / Fermiabs - is probably a good bet for this since
they are very good at supporting their own clusters. There is a problem, 
though in that compile time options may not be the same between RHEL and 
Scientific. ...

All part of life's rich tapestry ... 

> 
> 3) Docker/Container based:  Sort of a cross between stateless and VM
> based, it provides direct hardware access, and you can set up
> effectively independent and completely supportable containers to run
> on each system, independent of the OS requirements of the job.  See
> http://www.docker.com/whatisdocker/
> 

This may be useful for some users and some subsets of problems: here's hoping :)

> 
> >Being part of a larger community, running the same builds, has its
> >advantages.  You won't be the only person encountering a weird stability
> >or performance bug.  You also get vendor hardware support, which is
> >huge.
> 
> We switched our systems to Debian after we saw this.  We've been
> quite unhappy with some of the horrible broken-ness we've seen in
> the init system with {Red Hat|CentOS} 6.x for a while, and that
> legal change plus some nasty unfixable dracut stuff pushed us to
> better vistas.
> 

:) I seem to remember advocating a Debian alternative as long ago
as the Extreme Linux CD :)

> Err ... no.  The center of mass of the market has moved on to the
> faster changing distributions.  We opted for Debian over Ubuntu due
> to silliness in the Ubuntu kernel bits that made adding our patches
> hard. Much easier with a sane system.  Its very ... very ... hard to
> fix all the breakage when we make changes to CentOS/Red Hat.  You
> might say "don't change", but since part of our value is inherent in
> the changes, well ...
> 
> As I've been saying for more than a decade, the application OS
> requirements are a detail of the job.  Tools like kvm and docker let
> us get away from having a massive impedance mismatch between
> application requirements and node software environment requirements.
> 
> Having fought the supported OS battles for decades (jeez ... since
> the 80s!), and having the scars to prove it, I personally prefer the
> simpler/better/lower friction route.  No more square pegs in round
> holes.
> 

So much here to agree with.

All the best,

AndyC

> -- 
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics, Inc.
> email: landman at scalableinformatics.com
> web  : http://scalableinformatics.com
> twtr : @scalableinfo
> phone: +1 734 786 8423 x121
> cell : +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf