[Beowulf] Docker vs KVM paper by IBM

[Christopher Samuel]

On 29/01/15 05:32, Joe Landman wrote:

> Docker/VMs allow you to package your app, once, and be done with it. 
> New app, new package.  Packaging can be done programmatically.

This is the "appification" of HPC, à la mobile phones.

It brings with it the same issue that has dogged the distro maintainers
with things like the Chrome browser that bundles a whole heap of system
libraries - when a critical vulnerability appears in a library, how do
you make sure everything is fixed?

You can upgrade your base OS, but when a vulnerable container is run
again then that vulnerability is back.  Yes, theoretically[1] the impact
is constrained to the container (and any filesystems that it can see),
but if that job is running for a few weeks (or longer) that's a few
weeks (or longer) of vulnerability exposure you've got.

It also means rather than needing the space for a distro plus all the
HPC stack you put on there you end up with a copy of the distro and HPC
stack for each container.  Plus each version of the container.

I prefer the (now defunct I believe) Globus Workspace idea of having a
job that requests an OS version, a stack version and then supplies its
own config info and GW built the VM for you from that, and threw the VM
(but hopefully not your results) away at the end.

But in the end it's a balance of risk decision, weighing security needs
and exposure risk against predictability, the reproducibility of results
and provenance.

> Need a version of a library your sysadmin has told you will never be allowed on
> the system because its not distro approved?  Fine, container/VM-ize it.

I strongly suspect if the sysadmin won't install a library for you then
you're probably going to have a much harder time getting them to
consider containers. :-)

I've got to say I'm puzzled by the assumption that these things need to
affect the base OS.  We install the libraries, packages and other tools
that users want under /usr/local/$name/$version-$compiler and just use
modules to manage that.  It's visible across the whole cluster.

Want a different version of GCC?  We've got 4 different versions in
addition to the system GCC. If it's not there ask for it.  We've also
got Intel v13, v14 and v15 installed for users to pick and choose from.
Perl, Python? Check, one of the former, three of the latter.

We only have Open-MPI, but if people really needed MVAPICH or MPICH we
could get it but so far we've not been asked for it.  COMSOL uses
Intel MPI happily enough.

Our oldest Intel cluster has 398 modules available, our newer Intel
cluster has 293 modules and our BG/Q has 223.

That's counted via the slightly clumsy way of:

module avail 2>&1 | fgrep -v -- '---' | fgrep / |  sed -re 's/\s+/\n/' | wc -l

> Only your code/environment is at "risk."  Need to install HP-MPI (c'mon,
> we've all run into vendor apps that were built against one very esoteric
> version of the library ... ) to run your code?  Sure, do it in a container.

AFAIK Docker containers don't (by default) get access to devices, so
you'd need to negotiate with the sysadmin to agree to map things like
IB devices mapped into it.

Not impossible, but not available out of the box I suspect.

All the best,
Chris

[1] - whenever I use the term theoretical in relation to security I can't
help but remember Mudge et. al - "L0pht Heavy Industries. Making the
theoretical practical since 1992".

-- 
 Christopher Samuel        Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/      http://twitter.com/vlsci