<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>I'd be interested<br>
</p>
<pre class="moz-signature" cols="72">Prentice </pre>
<div class="moz-cite-prefix">On 9/23/21 10:37 AM, Pizarro, Angel via
Beowulf wrote:<br>
</div>
<blockquote type="cite"
cite="mid:C2E75C90-71E4-4BD0-A539-132CF07EC8A5@amazon.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}div.WordSection1
{page:WordSection1;}</style>
<div class="WordSection1">
<p class="MsoNormal">DISCLOSURE: I work for AWS HPC Developer
Relations in the services team. We developer AWS Batch, AWS
ParallelCluster, NICE DCV, etc.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Lambda’s limits today are 128MB to 10,240MB
(~10GB) and billed in 1MB per ms increments. 15 minute max
runtime for the function invocation. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Would you all be interested in a hands-on
self-paced workshop on creating (or porting) an application to
serverless environment? E.g. Monte-Carlo simulation, a genome
alignment or variant call, or some other problem? We have some
basic data processing documentation but nothing that speaks to
real-world HPC use case and that is a something I want to fill
the gap on if folks are interested in it. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Dr. Denis Bauer at CSIRO is also doing
interesting things with serverless.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-angel <o:p></o:p></p>
<p class="MsoNormal">-- <o:p></o:p></p>
<p class="MsoNormal">Angel Pizarro | Principal Developer
Advocate, HPC @ AWS <o:p>
</o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span
style="font-size:12.0pt;color:black">From: </span></b><span
style="font-size:12.0pt;color:black">Beowulf
<a class="moz-txt-link-rfc2396E" href="mailto:beowulf-bounces@beowulf.org"><beowulf-bounces@beowulf.org></a> on behalf of Guy
Coates <a class="moz-txt-link-rfc2396E" href="mailto:guy.coates@gmail.com"><guy.coates@gmail.com></a><br>
<b>Date: </b>Thursday, September 23, 2021 at 8:46 AM<br>
<b>To: </b>Tim Cutts <a class="moz-txt-link-rfc2396E" href="mailto:tjrc@sanger.ac.uk"><tjrc@sanger.ac.uk></a><br>
<b>Cc: </b>Beowulf <a class="moz-txt-link-rfc2396E" href="mailto:beowulf@beowulf.org"><beowulf@beowulf.org></a><br>
<b>Subject: </b>RE: [EXTERNAL] [Beowulf] Rant on why HPC
isn't as easy as I'd like it to be. [EXT]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<table class="MsoNormalTable" style="border-collapse:collapse"
cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr style="height:15.25pt">
<td style="width:842.35pt;border:solid #ED7D31
1.5pt;padding:0in 5.4pt 0in 5.4pt;height:15.25pt"
width="1123" valign="top">
<p><strong><span
style="font-family:"Calibri",sans-serif;color:black;background:#FFFF99">CAUTION</span></strong><span
style="color:black;background:#FFFF99">: This
email originated from outside of the organization.
Do not click links or open attachments unless you
can confirm the sender and know the content is
safe.</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Out of interest, how large are the
compute jobs (memory, runtime etc)? How easy to get them
to fit into a serverless environment?
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br>
Guy<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, 21 Sept 2021 at 13:02, Tim
Cutts <<a href="mailto:tjrc@sanger.ac.uk"
moz-do-not-send="true">tjrc@sanger.ac.uk</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">I think that’s exactly the
situation we’ve been in for a long time, especially in
life sciences, and it’s becoming more entrenched. My
experience is that the average user of our scientific
computing systems has been becoming less technically
savvy for many years now. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The presence of the cloud makes
that more acute, in particular because it makes it
easy for the user to effectively throw more hardware
at the problem, which reduces the incentive to make
their code particularly fast or efficient. Cost is
the only brake on it, and in many cases I’m finding
the PI doesn’t actually care about that. They care
that a result is being obtained (and it’s time to
first result they care about, not time to complete
all the analysis), and so they typically don’t have
much time for those of us who are telling them they
need to invest in time up front developing and
optimising efficient code.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">And cost is not necessarily the
brake I thought it was going to be anyway. One
recent project we’ve done on AWS has impressed me a
great deal. It’s not terribly CPU efficient, and
would doubtless, with sufficient effort, run much
more efficiently on premise. But it’s extremely
elastic in its nature, and so a good fit for the
cloud. Once a week, the project has to completely
re-analyse the 600,000+ COVID genomes we’e sequenced
so far, looking for new branches in the phylogenetic
tree, and to complete that analysis inside 8 hours.
Initial attempts to naively convert the HPC
implementation to run on AWS looked as though they
were going to be very expensive (~$50k per weekly
run). But a fundamental reworking of the entire
workflow to make it as cloud native as possible, by
which I mean almost exclusively serverless, has
succeeded beyond what I expected. The total cost is
<$5,000 a month, and because there is essentially
no statically configured infrastructure at all, the
security is fairly easy to be comfortable about.
And all of that was done with no detailed thinking
about whether the actual algorithms running in the
containers are at all optimised in a traditional HPC
sense. It’s just not needed for this particular
piece of work. Did it need software developers with
hardcore knowledge of performance optimisation?
No. Was it rapid to develop and deploy? Yes. Is
the performance fast enough for UK national COVID
variant surveillance? Yes. Is it cost effective?
Yes. Sold! The one thing it did need was
knowledgeable cloud architects, but the cloud
providers can and do help with that.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Tim<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">-- <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:black">Tim Cutts<br>
Head of Scientific Computing<br>
Wellcome Sanger Institute<o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote
style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On 21 Sep 2021, at 12:24,
John Hearns <<a
href="mailto:hearnsj@gmail.com"
target="_blank" moz-do-not-send="true">hearnsj@gmail.com</a>>
wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Some points well made
here. I have seen in the past job scripts
passed on from graduate student to graduate
student - the case I am thinking on was an
Abaqus script for 8 core systems, being run
on a new 32 core system. Why WOULD a
graduate student question a script given to
them - which works. They should be getting
on with their science. I guess this is where
Research Software Engineers come in.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Another point I would
make is about modern processor
architectures, for instance AMD
Rome/Milan. You can have different Numa
Per Socket options, which affect
performance. We set the preferred IO path
- which I have seen myself to have an
effect on latency of MPI messages. IF you
are not concerned about your hardware
layout you would just go ahead and run,
missing a lot of performance.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I am now going to be
controversial and common that over in
Julia land the pattern seems to be these
days people develop on their own laptops,
or maybe local GPU systems. There is a lot
of microbenchmarking going on. But there
seems to be not a lot of thought given to
CPU pinning or shat happens with
hyperthreading. I guess topics like that
are part of HPC 'Black Magic' - though I
would imagine the low latency crowd are
hot on them.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I often introduce
people to the excellent lstopo/hwloc
utilities which show the layout of a
system. Most people are pleasantly
surprised to find this.<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- The Wellcome Sanger Institute is
operated by Genome Research Limited, a charity
registered in England with number 1021457 and a
company registered in England with number 2742969,
whose registered office is 215 Euston Road, London,
NW1 2BE. <o:p></o:p></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
Beowulf mailing list, <a
href="mailto:Beowulf@beowulf.org" target="_blank"
moz-do-not-send="true">Beowulf@beowulf.org</a>
sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe)
visit <a
href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf"
target="_blank" moz-do-not-send="true">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Dr. Guy Coates<br>
+44(0)7801 710224 <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a>
</pre>
</blockquote>
</body>
</html>