[Beowulf] HPC workflows

[John Pellman]

>
> If HPC doesn't make it easy for these users to transfer their workflow
> to the cluster, and the cloud providers do, then the users will move
> to using the cloud even if it costs them 10%, 20% more because at the
> end of the day it is about getting the job done and not about spending
> time to work to antiquated methods of putting jobs in a cluster.
>


> And of course if the users would rather spend their department budgets
> with Amazon, Azure, Google, or others then the next upgrade cycle
> their won't be any money for the in house cluster...


Interestingly enough, Cornell has been adopting a sort of compromise
between traditional HPC and cloud computing by maintaining an
AWS-compatible private cloud on-prem (Red Cloud
<https://www.cac.cornell.edu/services/cloudservices.aspx>).  I'd speculate
that this would have the advantage of preventing researchers from "going
rogue" and foregoing traditional HPC groups entirely by going directly to
AWS.

On Tue, Nov 27, 2018 at 7:42 PM Gerald Henriksen <ghenriks at gmail.com> wrote:

> On Tue, 27 Nov 2018 07:51:06 -0500, you wrote:
>
> >On Mon, Nov 26, 2018 at 9:50 PM Gerald Henriksen <ghenriks at gmail.com>
> wrote:
> >> On Mon, 26 Nov 2018 16:26:42 +0100, you wrote:
> >> If on premise HPC doesn't change to reflect the way the software is
> >> developed today then the users will in the future prefer cloud HPC.
> >>
> >> I guess it is a brave new world for on premise HPC as far as that the
> >> users now, and likely more in the future, will have alternatives thus
> >> forcing the on premise HPC to "compete" in order to survive.
> >
> >this seems a bit too stringent of a statement for me.  i don't dismiss
> >or disagree with your premise, but i don't entirely agree that HPC
> >"must" change in order to compete.  We've all heard this kind of stuff
> >in the past if x doesn't change y will take over the world!
>
> HPC, like most things, exists to get something done.
>
> If HPC doesn't change to reflect the changes in society and the way
> the software is developed (*) then the users will look for more modern
> ways to replace traditional HPC.  As noted the software is no longer
> developed on workstations that are connected to the lab/company
> network but rather on laptops that stay with the user wherever they
> go.
>
> This in turn is at least in part what has driven to the rise of
> distributed version control, git in particular.
>
> If HPC doesn't make it easy for these users to transfer their workflow
> to the cluster, and the cloud providers do, then the users will move
> to using the cloud even if it costs them 10%, 20% more because at the
> end of the day it is about getting the job done and not about spending
> time to work to antiquated methods of putting jobs in a cluster.
>
> And of course if the users would rather spend their department budgets
> with Amazon, Azure, Google, or others then the next upgrade cycle
> their won't be any money for the in house cluster...
>
>
> * - note the HPC isn't unique in this regard.  The Linux distributions
> are facing their own version of this, where much of the software is no
> longer packagable in the traditional sense as it instead relies on
> language specific packaging systems and languages that don't lend
> themselves to the older rpm/deb style system.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20181128/b57a8ef2/attachment-0001.html>