[Beowulf] Docker vs KVM paper by IBM

[Joe Landman]

On 01/28/2015 02:08 PM, Ellis H. Wilson III wrote:
> On 01/28/2015 02:00 PM, Gavin W. Burris wrote:
>> Researchers do not want to be sysadmins, and if they do, they
>> aren't publishing.
>
> This statement is so infuriatingly incorrect I'm embarrassed to even 
> be responding to it.

Heh ...

>
> You do remember this is the Beowulf list, originally comprised of 
> researchers who decided to be their own sysadmins/hardware vendors/etc 
> in order to get their research done, right?
>
> Even in today's mainly post-big-iron age, there are plenty of 
> systems-level CS researchers (like I was not more than 6 months back) 
> who ran their own clusters simply to get their work done. No, I really 
> don't feel like working with my IT staff every time I want to change 
> the toolchain or recompile such-and-such magical cache replacement 
> algorithm into this version of the kernel. Imagine that.  I just want 
> to do it.

This is in large part what is driving the whole concept.  Its hard 
enough to get updated drivers into specific types of clusters (the IT 
clusters commonly run with RHEL variants) ... imagine trying to replace 
a system level package for something that you need to support your 
work.  You don't need to be negotiating with the IT staff to make this 
change.

FWIW:  this is one of the reasons we built our systems as being entirely 
PXE booted, and "auto configuring" upon startup.  Make it so that if you 
want to/have to make changes you can do it in a completely 
programmatically defined manner.  Make the substrate OS a simple 
function of the job launch.  Though Docker will allow us to simplify 
this even more ... our substrate can be that much simpler ...

... but this said, the OS/distro should not be an impediment to running 
code.  That Docker emerged and has caught fire is a testament to the 
fact that it generally *is* an impediment, and its sadly rare that you 
find things that just work right, out of the box/tarball without a 
dependency chain from hell to chase.

Large dependency radii suck. Docker (and VMs) let you contain this.

>
> I have no doubt there are tons of point-and-click researchers out 
> there who don't want to be sysadmins.  But making such broad strokes 
> on this list is asking for flames.

+1

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
e: landman at scalableinformatics.com
w: http://scalableinformatics.com
t: @scalableinfo
p: +1 734 786 8423 x121
c: +1 734 612 4615