[Beowulf] interesting article on HPC vs evolution of 'big data' analysis

[Gus Correa]

For those old enough to have heard somebody as great as
John Lennon sing that "the dream is over",
and rather mediocre "philosophers" claim the "end of history", 
prophecies that were never confirmed,
reading bombastic claims that MPI is dead is not so unsettling.
After all, reports of the death of Fortran, like Mark Twain's death,
have been greatly exaggerated.
It continues to hang on, outlived his creators,
and shows no sign of dying anytime soon.

Is Exascale really needed?
If so, would the hardware paradigm have to change also perhaps?
(Good scaling properties and reliability are problems not only for
software, are they?)
Why can't MPI adapt to the new scenario?

Gus Correa

On 04/08/2015 03:57 PM, Scott Atchley wrote:
> There is concern by some and outright declaration by others (including
> hardware vendors) that MPI will not scale to exascale due to issues like
> rank state growing too large for 10-100 million endpoints, lack of
> reliability, etc. Those that make this claim then offer up their
> favorite solution (a PGAS variant, Chapel, Legion, Open Community
> Runtime). Several assert that the event-driven/task-driven runtimes will
> take care of data partitioning, data movement, etc. and that the user
> only has to define relationships and dependencies while exposing as much
> parallelism as possible.
>
> The domain scientists shudder at the thought of rewriting existing
> codes, some of which have existed for decades. If they do get funding to
> rewrite, which new programming model should they pick? At this point,
> there is no clear favorite.
>
>
>
> On Wed, Apr 8, 2015 at 2:31 PM, Prentice Bisbal
> <prentice.bisbal at rutgers.edu <mailto:prentice.bisbal at rutgers.edu>> wrote:
>
>     I got annoyed by this article and had to stop reading it. I'll go
>     back later and try to give it a proper critique, but obviously
>     disagree with most of what I've read so far. Right of the bat, the
>     author implies that Big Data = HPC, and I disagree with that.
>
>     More ranting to come....
>
>     Prentice
>
>
>     On 04/08/2015 01:16 PM, H. Vidal, Jr. wrote:
>
>         Curious as to what the body of thought is here on this article:
>
>         http://www.dursi.ca/hpc-is-__dying-and-mpi-is-killing-it/
>         <http://www.dursi.ca/hpc-is-dying-and-mpi-is-killing-it/>
>
>         _________________________________________________
>         Beowulf mailing list, Beowulf at beowulf.org
>         <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>         To change your subscription (digest mode or unsubscribe) visit
>         http://www.beowulf.org/__mailman/listinfo/beowulf
>         <http://www.beowulf.org/mailman/listinfo/beowulf>
>
>
>     _________________________________________________
>     Beowulf mailing list, Beowulf at beowulf.org
>     <mailto:Beowulf at beowulf.org> sponsored by Penguin Computing
>     To change your subscription (digest mode or unsubscribe) visit
>     http://www.beowulf.org/__mailman/listinfo/beowulf
>     <http://www.beowulf.org/mailman/listinfo/beowulf>
>
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>