[Beowulf] Open source and the Draft Report of the Task Force on High Performance Computing

[atchley tds.net]

On Thu, Aug 28, 2014 at 9:31 AM, Gavin W. Burris <bug at wharton.upenn.edu>
wrote:

> Hi, C.
>
> Yes, there are many closed source, domain-specific, proprietary tools, and
> I
> like them!  Any full HPC stack has many open source pieces as functional
> components.  One cannot pick what does or does not count.
>
> To this reports defense, it does read as if to reference the near future, a
> future where we throw out all existing code and start from scratch.  The
> report
> seems to be making a sci-fi-esque call for a complete rethink and
> reinvention
> of billions-and-billions of lines of code.  Very admirable.
>
> Cheers.


I do not read it as a call to throw out everything. DOE plans to support
MPI+X (where X can be OpenMP or OpenACC or something else) for the
foreseeable future. Asking scientists to rewrite from scratch is a tall
order. The problem with MPI+X now is that we are tuning for specific
machines at specific sites. It is no longer portable solution by itself
(i.e. write it once and run everywhere and expect the best performance).

Some prefer a PGAS model and are looking at it as a hedge against MPI
scaling issues for very large system. The thinking is, that with millions
of nodes, that MPI rank lookup information becomes to consume more memory
than we want.

Still others are proposing event-driven task models (EDTs) such as Legion
and the Open Community Runtime. These promise the ability to write once and
let the compiler and runtime extract the best performance from the given
hardware. The downside is the complete rewrite and the difficulty to
diagnose poor performance (i.e. did I do a bad job or was it the compiler's
or runtime's fault).

To DOE's credit, they are not picking and choosing. They are funding R&D
such as Fast Forward and Design Forward as well as software development.
They _do_ want to see the investments pay off and be used.

Scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20140828/533ebe39/attachment.html>