[Beowulf] HPC workflows

[John Hanks]

I think you do a better job explaining the underpinnings of my frustration
with it all, but then arrive at a slightly different set of conclusions.
I'd be the last to say autotools isn't complex, in fact pretty much all
build systems eventually reach an astounding level of complexity. But I'm
not sure copy/paste of an autotools recipe is any more egregious than a
copy/paste of cmake or any others. That seems to be the first step to
learning how anything works, so I accept that it'll happen a lot with
anything. Credit for a huge portion of my limited success is owed to
copy/paste from stack overflow of things I didn't understand at first, so I
can't really throw rocks at the practice. People tend to dig as deep as
they need to to get something to work and then wander off unless they are
getting paid to keep digging or just like a particular hole.

Certainly the inability of distros to find the person-hours to package
everything plays a role as well, your cause and effect chain there is
pretty accurate. Where I begin to branch is at the idea of software that is
unable to be packaged in an rpm/deb. This is where our collective computing
train goes off the rails. Reaching a point where something is too
complicated to package with a set of tools which have the ability to run
*any arbitrary set of commands* and concluding the solution is to invent a
new set of tools/abstractions to run *any arbitrary set of commands* is the
derailing step. Worse, with containers that set of arbitrary commands
generally starts out by running "apt-get install ...", a precarious
dependency for an abstraction layer whose primary claim to fame is getting
past the limitations of those same commands.

I'll rephrase my earlier complaint about the rise of these abstractions as
"I can't figure out how to write spec files so I'll create a new
build/distribution system" which is, basically, a variation on the classic
argument from ignorance. Maybe that has merit if these new tools turned out
to actually be simpler and easier, but to date that hasn't been the case.
The water just keeps getting muddier.

The thing we can never measure and thus can only speculate about forever
is:  if all the person-hours poured into containers (and pypi/pip and cran
and cpan and maven and scons and ...) had been poured into rpm/deb
packaging would we just be simply apt/yum/dnf installing what we needed
today? (I'm ignoring other OS/packaging tools, but you get the idea.) We
can't run that experiment, but I suspect that it isn't a limitation of
rpm/deb to be able to package so much as it is that there is no incentive
to package in rpm/deb, but there is an incentive to invent/monetize new
abstraction layers. It doesn't hurt that humans crave novelty, so new is
always more appealing than old and without a good grasp of the old it's
impossible to properly evaluate the new relative to it. How does the old
quote go, "those who cannot remember the past are condemned to repeat it."
I look forward to ever more complex methods to package containers once we
have containers that are too complex to deliver as containers. Our only
real hope is that eventually human language will run out of metaphors to
use when monetizing the next big abstraction.

griznog

On Tue, Dec 4, 2018 at 9:51 PM Gerald Henriksen <ghenriks at gmail.com> wrote:

> On Mon, 3 Dec 2018 10:12:10 -0800, you wrote:
>
> > And then I realized that I was seeing
> >software which was "easier to containerize" and that "easier to
> >containerize" really meant "written by people who can't figure out
> >'./configure; make; make install' and who build on a sand-like foundation
> >of fragile dependencies to the extent that it only runs on their Ubuntu
> >laptop so you have to put their Ubuntu laptop in a container."
>
> The problem is that essentially nobody knows how autotools works, so
> that those C/Fortran codes that use it have usually copy/pasted
> something until it seems to work.
>
> So 2 things happened.
>
> First, all the non-traditional languages created their own build
> systems, and more importantly their own package management systems.
> This developed because most development was happening on non-Linux
> systems, because Linux still struggles on laptops and laptops have
> taken over the non-server computer world.  It also happened because
> those developers using Linux, or at least aware of deploying on Linux,
> rebelled at the limitations of the Linux ecosystem (namely
> libraries/components that hadn't been natively packaged, or the normal
> conflict of the "wrong" version being packaged).
>
> A side effect of all these package management systems is that they are
> frequently hostile to the "Linux way", and create software that is
> essentially unable to be packaged into RPM or deb format.
>
> The other issue of course is that open source won, and the explosion
> of open source means the distributions no longer have the person-power
> not just to package everything, but for those packages to do much of
> the heavy lifting in keeping the software up to date.
>
> As for autotools, it to is now being abandoned with the 2 leading
> contenders being cmake and meson, but it being C++ the chaos wouldn't
> be complete with multiple competing package management solutions...
>
> > Then I
> >started asking myself "do I want to trust software of that quality?" And
> >after that, "do I want to trust the tools written to support that type of
> >poor-quality software?"
>
> On the other hand can you really trust the software built in more
> traditional ways? see OpenSSL / Heartbleed.
>
> >From the perspective of the software being containerized, I'm even more
> >skeptical. In my world (bioinformatics) I install a lot of crappy
> software.
> >We're talking stuff resulting from "I read the first three days of 'learn
> >python in 21 days' and now I'm an expert, just run this after installing
> >these 17 things from pypi...and trust the output" I'm good friends with
> >crappy software, we hang out together a lot. To me it just doesn't feel
> >like making crappy software more portable is the *right* thing to do. When
> >I walk my dog, I follow him with a bag and "containerize" what drops out.
> >It makes it easier to carry around, but doesn't change what it is. As of
> >today I see the biggest benefit of containers as that they force a
> >developer to actually document the install procedure somewhere in a way
> >that actually has to work so we can see firsthand how ridiculous it is
> >(*cough* tensorflow *cough*).
>
> All very true.  To paraphrase, containers are the best of a bunch of
> bad options.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20181205/aad72719/attachment-0001.html>