[Beowulf] Docker in HPC
landman at scalableinformatics.com
Fri Nov 29 07:49:04 PST 2013
On 11/29/2013 06:16 AM, Olli-Pekka Lehto wrote:
> Thus it might not be a question of choosing to adopt it and sell it to
> the users but rather having to respond to user demand for it. This means
> understanding and dealing with the constraints involved with especially
> InfiniBand-based HPC clusters which you outlined pretty well in your mail.
My thoughts are, that while its not necessarily the first to market with
the concept, the Docker.io folks have benefited from a technological
convergence of sorts, where they were able to assemble, from
pre-existing parts (that work relatively well on their own), the
solution. Moreover, there are real problems that this can be applied
to, with the specifics of repeatable environments for computational
Actually that in and of itself bears a strong need for discussion.
Non-repeatable results, or results that cannot be repeated due to an
inability to run or rerun the simulation, should be treated as transient
results, and handled with the appropriate skepticism that infrequent
results are. Or if the investigator wants to leverage some of the
statistical techniques used in HEP, there are techniques for doing so.
But it is important to, not simply publish papers on research results,
but code/algorithms, and raw data if possible.
I remember having my hands proverbially slapped when I wanted to do this
in the early 90s during my thesis work. It has always bugged me ... I
benefited from shared and peer reviewed code and data. I would hope
that there would be an appropriate journal for that someday.
Mebbe there is, and I don't know about it now. If not, this could be a
fun side project. Sort of a PLOS code/data/algorithm/test case
repository to reference published work. Not bio focused specifically
(PLOS has been very bio centric). I know there was a Computational
Physics Communications journal (mostly CFD and related) 20 years ago.
And one of the very first things I did for my thesis group was to type
in the Hermann Skillman code
which was published in 1963 in ancient fortran, and try to port it to
(then) modern fortran. I wound up writing my own self-consistent field
electronic structure calculation, but I couldn't share it as it had
Numerical Recipes code in it (I was lazy and used some splines, and the
FWIW: I still pull that code out every few years and look at it. Mebbe
I'll rewrite it someday. I dunno.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
twtr : @scalableinfo
phone: +1 734 786 8423 x121
cell : +1 734 612 4615
More information about the Beowulf