[Beowulf] Docker in HPC

[Joe Landman]

On 11/29/2013 06:16 AM, Olli-Pekka Lehto wrote:

[...]

> Thus it might not be a question of choosing to adopt it and sell it to
> the users but rather having to respond to user demand for it. This means
> understanding and dealing with the constraints involved with especially
> InfiniBand-based HPC clusters which you outlined pretty well in your mail.

My thoughts are, that while its not necessarily the first to market with 
the concept, the Docker.io folks have benefited from a technological 
convergence of sorts, where they were able to assemble, from 
pre-existing parts (that work relatively well on their own), the 
solution.  Moreover, there are real problems that this can be applied 
to, with the specifics of repeatable environments for computational 
studies ...

Actually that in and of itself bears a strong need for discussion. 
Non-repeatable results, or results that cannot be repeated due to an 
inability to run or rerun the simulation, should be treated as transient 
results, and handled with the appropriate skepticism that infrequent 
results are.  Or if the investigator wants to leverage some of the 
statistical techniques used in HEP, there are techniques for doing so.

But it is important to, not simply publish papers on research results, 
but code/algorithms, and raw data if possible.

I remember having my hands proverbially slapped when I wanted to do this 
in the early 90s during my thesis work.  It has always bugged me ... I 
benefited from shared and peer reviewed code and data.  I would hope 
that there would be an appropriate journal for that someday.

Mebbe there is, and I don't know about it now.  If not, this could be a 
fun side project.  Sort of a PLOS code/data/algorithm/test case 
repository to reference published work.  Not bio focused specifically 
(PLOS has been very bio centric).   I know there was a Computational 
Physics Communications journal (mostly CFD and related) 20 years ago. 
And one of the very first things I did for my thesis group was to type 
in the Hermann Skillman code 
(http://hermes.phys.uwm.edu/projects/elecstruct/hermsk/HS.Overview.html) 
which was published in 1963 in ancient fortran, and try to port it to 
(then) modern fortran.  I wound up writing my own self-consistent field 
electronic structure calculation, but I couldn't share it as it had 
Numerical Recipes code in it (I was lazy and used some splines, and the 
Runge-Kutta integrator).

FWIW: I still pull that code out every few years and look at it.  Mebbe 
I'll rewrite it someday.  I dunno.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
twtr : @scalableinfo
phone: +1 734 786 8423 x121
cell : +1 734 612 4615