[Beowulf] HPC workflows

[Gerald Henriksen]

On Mon, 3 Dec 2018 10:12:10 -0800, you wrote:

> And then I realized that I was seeing
>software which was "easier to containerize" and that "easier to
>containerize" really meant "written by people who can't figure out
>'./configure; make; make install' and who build on a sand-like foundation
>of fragile dependencies to the extent that it only runs on their Ubuntu
>laptop so you have to put their Ubuntu laptop in a container."

The problem is that essentially nobody knows how autotools works, so
that those C/Fortran codes that use it have usually copy/pasted
something until it seems to work.

So 2 things happened.

First, all the non-traditional languages created their own build
systems, and more importantly their own package management systems.
This developed because most development was happening on non-Linux
systems, because Linux still struggles on laptops and laptops have
taken over the non-server computer world.  It also happened because
those developers using Linux, or at least aware of deploying on Linux,
rebelled at the limitations of the Linux ecosystem (namely
libraries/components that hadn't been natively packaged, or the normal
conflict of the "wrong" version being packaged).

A side effect of all these package management systems is that they are
frequently hostile to the "Linux way", and create software that is
essentially unable to be packaged into RPM or deb format.

The other issue of course is that open source won, and the explosion
of open source means the distributions no longer have the person-power
not just to package everything, but for those packages to do much of
the heavy lifting in keeping the software up to date.

As for autotools, it to is now being abandoned with the 2 leading
contenders being cmake and meson, but it being C++ the chaos wouldn't
be complete with multiple competing package management solutions...

> Then I
>started asking myself "do I want to trust software of that quality?" And
>after that, "do I want to trust the tools written to support that type of
>poor-quality software?"

On the other hand can you really trust the software built in more
traditional ways? see OpenSSL / Heartbleed.

>From the perspective of the software being containerized, I'm even more
>skeptical. In my world (bioinformatics) I install a lot of crappy software.
>We're talking stuff resulting from "I read the first three days of 'learn
>python in 21 days' and now I'm an expert, just run this after installing
>these 17 things from pypi...and trust the output" I'm good friends with
>crappy software, we hang out together a lot. To me it just doesn't feel
>like making crappy software more portable is the *right* thing to do. When
>I walk my dog, I follow him with a bag and "containerize" what drops out.
>It makes it easier to carry around, but doesn't change what it is. As of
>today I see the biggest benefit of containers as that they force a
>developer to actually document the install procedure somewhere in a way
>that actually has to work so we can see firsthand how ridiculous it is
>(*cough* tensorflow *cough*).

All very true.  To paraphrase, containers are the best of a bunch of
bad options.