[Beowulf] HPC workflows

[Eliot Eshelman]

Those interested in providing user-friendly HPC might want to take a 
look at Open OnDemand. I'm not affiliated with this project, but wanted 
to make sure it got a plug. I've heard good things so far.

http://openondemand.org/

Eliot



On 11/26/18 10:26, John Hearns via Beowulf wrote:
> This may not be the best place to discuss this - please suggest a better
> forum if you have one.
> I have come across this question in a few locations. Being specific, I am a
> fan of the Julia language. Ont he Juia forum a respected developer recently
> asked what the options were for keeping code developed on a laptop in sync
> with code being deployed on an HPC system.
> There was some discussion of having Git style repositories which can be
> synced to/from.
> My suggestion was an ssh mount of the home directory on the HPC system,
> which I have configured effectively int he past when using remote HPC
> systems.
>
> At a big company I worked with recently, the company provided home
> directories on NFS Servers. But the /home/username directory on the HPC was
> different - on higher performance storage. The 'company' home was mounted -
> so you could copy between them. But we did have the inevitable incidents of
> jobs being run from company NFS - and pulling code across the head node
> interfaces etc.
>
> Developers these days are used to carrying their Mac laptops around and
> working at hotdesks, at home, at conferences. ME too - and I love it.
> Though I have a lovely HP Spectre Ultrabook.
> Again their workflow is to develop on the laptop and upload code to Github
> type repositories. Then when running on a cloud service the software ids
> downloaded from the Repo.
> There are of course HPC services on the cloud, with gateways to access them.
>
> This leads me to ask - shoudl we be presenting HPC services as a 'cloud'
> service, no matter that it is a non-virtualised on-premise setup?
> In which case the way to deploy software would be via downloading from
> Repos.
> I guess this is actually more common nowadays.
>
> I think out loud that many HPC codes depend crucially on a $HOME directory
> being presnet on the compute nodes as the codes look for dot files etc. in
> $HOME. I guess this can be dealt with by fake $HOMES which again sync back
> to the Repo.
>
> And yes I know containerisation may be the saviour here!
>
> Sorry for a long post.
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20181128/bf21f753/attachment.html>