[Beowulf] HPC workflows

[Gerald Henriksen]

On Wed, 5 Dec 2018 09:35:07 -0800, you wrote:

>Certainly the inability of distros to find the person-hours to package
>everything plays a role as well, your cause and effect chain there is
>pretty accurate. Where I begin to branch is at the idea of software that is
>unable to be packaged in an rpm/deb.

In some convenient timing, the following was posted by overtmind on
Reddit discussing why Atom hasn't been packaged for Fedora(*):

---
"This means, for every nodejs dependency Electron needs - and there
are a metric #$%# ton - since you can't use npm as an
installer/package manager - you need to also package all of those and
make sure they're in fedora and up-to-date, and then you also need to
package all of the non-nodejs dependencies that come along with
Electron apps, such as electron itself, and THEN you need to extract
and remove all of the vendor'd libraries and binaries that essentially
make Electron work, and THEN you need to make sure that there's no
side-car'd non-free or questionable software that is forbidden in
fedora also, like ffmpeg. G'head and look at the Chromium SPEC, it's a
living nightmare (Spot godbless your heart)"
---

Now obviously you could do what for example Java does with a jar file,
and simply throw everything into a single rpm/deb and ignore the
packaging guidelines, but then you are back to in essence creating a
container and just hiding it behind a massive rpm/deb.

>The thing we can never measure and thus can only speculate about forever
>is:  if all the person-hours poured into containers (and pypi/pip and cran
>and cpan and maven and scons and ...) had been poured into rpm/deb
>packaging would we just be simply apt/yum/dnf installing what we needed
>today? (I'm ignoring other OS/packaging tools, but you get the idea.)

I (theoretically) could write a new library in
Python/Perl/Javascript/Go/etc. and with very minimal effort can place
that library in the repository for that language with minimal effort.
My library is now available to everyone using that language regardless
of what OS they are using.

Alternately, I could spend many, many hours perhaps even days learning
multiple different packaging systems, joining multiple different
mailing lists / bugzillas / build systems, so that I can make my
library easily available to people on Windows, macOS, Fedora, RHEL,
openSUSE, Debian, Ubuntu, ...  - or alternately hope that someone will
not only take the time to package my library for all those different
platforms, but also commit the future time to keep it up to date.

Option 2 worked 20 years ago when we only cared about 2 or 3
distributions of Linux and had a lot less open source / free software.
But, unfortunately, it does not scale and so for that reason (and a
few others) the effort to create Docker / npm, maven, etc. is the
lesser of the options.

* - https://www.reddit.com/r/Fedora/comments/a3q1a2/atom_editoride/