<div dir="ltr">It seems that in environments where you don't care about security then docker is a great enabler so that scientists can make any kind of mess in a sandbox type environment and no one cares because your not on a public facing network. There are however difficulties in using docker with mpi so its probably not suitable for clustering yet.<div><br></div><div>For people requiring security then docker does not provide any added protection or containerisation and, in fact could be an impediment to building secure systems. Redhat have done some interesting work here by integrating Docker with SELinux so that each container runs in its own SELinux domain.</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 29 January 2015 at 02:10, Joe Landman <span dir="ltr"><<a href="mailto:landman@scalableinformatics.com" target="_blank">landman@scalableinformatics.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><br>
On 01/28/2015 07:02 PM, Christopher Samuel wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 29/01/15 05:32, Joe Landman wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Docker/VMs allow you to package your app, once, and be done with it.<br>
New app, new package. Packaging can be done programmatically.<br>
</blockquote>
This is the "appification" of HPC, à la mobile phones.<br>
<br>
It brings with it the same issue that has dogged the distro maintainers<br>
with things like the Chrome browser that bundles a whole heap of system<br>
libraries - when a critical vulnerability appears in a library, how do<br>
you make sure everything is fixed?<br>
</blockquote>
<br></span>
This is an issue, but its an issue with distros as well. If you have a build process for your containers, slotting in the patches from the mailing lists shouldn't be hard. If you have a distro, you have to wait for the official package from the distro maintainer.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
You can upgrade your base OS, but when a vulnerable container is run<br>
again then that vulnerability is back. Yes, theoretically[1] the impact<br>
is constrained to the container (and any filesystems that it can see),<br>
but if that job is running for a few weeks (or longer) that's a few<br>
weeks (or longer) of vulnerability exposure you've got.<br>
</blockquote>
<br></span>
While true as docker containers are effectively immutable, you only expose the ports you need exposed from the containers. Since the containers are immutable, you can't "infect" the storage, only the memory. But ... as with everything, you get nothing for free and there are engineering choices to be made for this.<br>
<br>
See this: <a href="http://devops.com/blogs/docker-leaving-immutable-infrastructure-2/" target="_blank">http://devops.com/blogs/<u></u>docker-leaving-immutable-<u></u>infrastructure-2/</a><br>
<br>
For long running apps, the concept of a state-saving and graceful shutdown/restart is pretty important. There have been attempts at this via checkpointing apps that snapshot state to disk, that you can resume later. This concept won't work well (that is, binary state) for containers whose core parts change between checkpoint and restart. It will work fine if you can provide a restart type mode and save state so as to resume from a specific point without forcing all the binary state preservation.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
It also means rather than needing the space for a distro plus all the<br>
HPC stack you put on there you end up with a copy of the distro and HPC<br>
stack for each container. Plus each version of the container.<br>
</blockquote>
<br></span>
For a centralized storage of containers, this isn't so much of an issue. For a distro, it does mean you can pare down un-needed packages. For the HPC stack, it means you can keep much of it lean and mean.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I prefer the (now defunct I believe) Globus Workspace idea of having a<br>
job that requests an OS version, a stack version and then supplies its<br>
own config info and GW built the VM for you from that, and threw the VM<br>
(but hopefully not your results) away at the end.<br>
</blockquote>
<br></span>
Way way back in the day I wrote this little tool called DragonFly (see <a href="https://git.scalableinformatics.com/?p=DragonFlyCloud.git;a=summary" target="_blank">https://git.<u></u>scalableinformatics.com/?p=<u></u>DragonFlyCloud.git;a=summary</a> and I've not moved it over to gitlab/github yet). See <a href="https://scalability.org/?p=425" target="_blank">https://scalability.org/?p=425</a> and <a href="https://scalability.org/?p=482" target="_blank">https://scalability.org/?p=482</a> . It did this (albeit on a fixed configuration cluster). It generated the run script from an xml description of the run, on the local hardware. I had it do data transfers, and all manner of interesting things. I could have easily added in the OS spec as well had I an ability to make the OS version a detail of the job (which, curiously, I do now).<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
But in the end it's a balance of risk decision, weighing security needs<br>
and exposure risk against predictability, the reproducibility of results<br>
and provenance.<br>
</blockquote>
<br></span>
Most HPC codes aren't a major risk. Though as we get into big data of unknown provenance, the IoT and IoH (internet of hacks) this is more of an issue. Immutable containers are a good start in some ways. Restartable apps are another good thing. Apps that provide a mechanism to restart by providing enough state to make this easy is something beneficial.<div class="HOEnZb"><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Need a version of a library your sysadmin has told you will never be allowed on<br>
the system because its not distro approved? Fine, container/VM-ize it.<br>
</blockquote>
I strongly suspect if the sysadmin won't install a library for you then<br>
you're probably going to have a much harder time getting them to<br>
consider containers. :-)<br>
<br>
I've got to say I'm puzzled by the assumption that these things need to<br>
affect the base OS. We install the libraries, packages and other tools<br>
that users want under /usr/local/$name/$version-$<u></u>compiler and just use<br>
modules to manage that. It's visible across the whole cluster.<br>
<br>
Want a different version of GCC? We've got 4 different versions in<br>
addition to the system GCC. If it's not there ask for it. We've also<br>
got Intel v13, v14 and v15 installed for users to pick and choose from.<br>
Perl, Python? Check, one of the former, three of the latter.<br>
<br>
We only have Open-MPI, but if people really needed MVAPICH or MPICH we<br>
could get it but so far we've not been asked for it. COMSOL uses<br>
Intel MPI happily enough.<br>
<br>
Our oldest Intel cluster has 398 modules available, our newer Intel<br>
cluster has 293 modules and our BG/Q has 223.<br>
<br>
That's counted via the slightly clumsy way of:<br>
<br>
module avail 2>&1 | fgrep -v -- '---' | fgrep / | sed -re 's/\s+/\n/' | wc -l<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Only your code/environment is at "risk." Need to install HP-MPI (c'mon,<br>
we've all run into vendor apps that were built against one very esoteric<br>
version of the library ... ) to run your code? Sure, do it in a container.<br>
</blockquote>
AFAIK Docker containers don't (by default) get access to devices, so<br>
you'd need to negotiate with the sysadmin to agree to map things like<br>
IB devices mapped into it.<br>
<br>
Not impossible, but not available out of the box I suspect.<br>
<br>
All the best,<br>
Chris<br>
<br>
[1] - whenever I use the term theoretical in relation to security I can't<br>
help but remember Mudge et. al - "L0pht Heavy Industries. Making the<br>
theoretical practical since 1992".<br>
<br>
</blockquote>
<br>
-- <br></div></div><span class="im HOEnZb">
Joseph Landman, Ph.D<br>
Founder and CEO<br>
Scalable Informatics, Inc.<br>
e: <a href="mailto:landman@scalableinformatics.com" target="_blank">landman@scalableinformatics.<u></u>com</a><br>
w: <a href="http://scalableinformatics.com" target="_blank">http://scalableinformatics.com</a><br>
t: @scalableinfo<br>
p: <a href="tel:%2B1%20734%20786%208423%20x121" value="+17347868423" target="_blank">+1 734 786 8423 x121</a><br>
c: <a href="tel:%2B1%20734%20612%204615" value="+17346124615" target="_blank">+1 734 612 4615</a><br>
<br></span><div class="HOEnZb"><div class="h5">
______________________________<u></u>_________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/<u></u>mailman/listinfo/beowulf</a><br>
</div></div></blockquote></div><br></div>