[Beowulf] [EXTERNAL] Re: ***UNCHECKED*** Re: Spark, Julia, OpenMPI etc. - all in one place

[Oddo Da]

Jim, all your replies in the three separate emails - make valid points and
give me some food for thought. Thanks.

On Wed, Oct 14, 2020 at 3:10 PM Lux, Jim (US 7140) <james.p.lux at jpl.nasa.gov>
wrote:

>
>
>
>
> *From: *Beowulf <beowulf-bounces at beowulf.org> on behalf of Oddo Da <
> oddodaoddo at gmail.com>
> *Date: *Wednesday, October 14, 2020 at 11:09 AM
> *To: *Michael Di Domenico <mdidomenico4 at gmail.com>
> *Cc: *"beowulf at beowulf.org" <beowulf at beowulf.org>
> *Subject: *[EXTERNAL] Re: [Beowulf] ***UNCHECKED*** Re: Spark, Julia,
> OpenMPI etc. - all in one place
>
>
>
> On Wed, Oct 14, 2020 at 1:24 PM Michael Di Domenico <
> mdidomenico4 at gmail.com> wrote:
>
> On Wed, Oct 14, 2020 at 11:53 AM Oddo Da <oddodaoddo at gmail.com> wrote:
>
> i'll agree that in some respects software engineering has gotten
> better in the last 20yrs, but it's subjective.  there are a lot of
> things that have gotten better and there are a lot of things that are
> much worse.  but i'm not sure you can apply that statement to HPC.
> HPC code doesn't churn like business code or even more volatile cloud
> code.  HPC code is usually written to solve something specific and
> gets incremental updates over time.  usually that something specific
> hasn't changed the last 20yrs (think physics/chemistry) the models we
> use to describe or solve the problems likely have, but the underlying
> code is probably basically the same with tweaks along the way to fit
> the new model.
>
>
>
> I disagree. I think yes, there is old code that does not churn but there
> are always new people/grad students coming into the field. They too are
> being pointed in the same direction of how to do things, which is what we
> are discussing here ;-)
>
>
>
>
>
>
>
> Yes, there are new people coming in.  But they’re not developing new
> modeling codes from scratch – they’re typically “improving” the existing
> codes.   And as Michael pointed out, there’s significant resistance to
> change when you’ve got a code base that is known, debugged, and has known
> warts.    Big changes occur when a modeling paradigm shift occurs.  And
> those are not real common.
>
>
>
>
>
>
>
>
>
>
>
> an evolution to MPI.  but it goes back to technical debt.  to re-write
> something in chapel is non-trivial and may not be worth the time.
> writing something new and choosing chapel is really left up to the
> developer.  i have some chapel users here and there, but they're a
> minority.  and since chapel is largely only found on cray machines its
> exposure is low
>
>
>
> It seems that in your world nothing new ever gets written? You are talking
> only about re-writes ;).
>
>
>
> For many HPC science applications this is true – Physics models change
> very slowly. Once you have a gridded finite element model, it “just works”
> and there’s not huge demand for new modeling approaches.
>
>
>
> Changes in language usage usually occur because of a technical problem
> with the existing code that means it just cannot be modified.  More than
> one ambitious person has said “let’s re-do program X from Fortran to C++ or
> Java or Ada or Python” and found it a bigger challenge than expected.
> There is a definite preference for the devil you know.
>
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://beowulf.org/pipermail/beowulf/attachments/20201014/64453599/attachment.html>