<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>The problem with timeslicing is that when one job is pre-empted,
it's state needs to be stored somewhere so the next job can run.
Since many HPC jobs are memory intensive, using RAM for this is
not usually an option. Which leaves writing the state to disk.
Since disk is many orders of magnitude slower than RAM, writing
state to disk for timeslicing would ultimately reduce the
throughput of the cluster. It's much more efficient to have one
job "own" the nodes until it completes.</p>
<p>Yes, jobs do checkpointing, but I'm assuming the checkpointing
isn't happening as frequently as your proposed timeslicing, and
that checkpointing isn't writing the entire state to disk. <br>
</p>
<pre class="moz-signature" cols="72">Prentice </pre>
<div class="moz-cite-prefix">On 1/17/20 12:35 AM, Lux, Jim (US 337K)
via Beowulf wrote:<br>
</div>
<blockquote type="cite"
cite="mid:649320D1-FC7E-43FA-948F-C0BC5228EF1D@jpl.caltech.edu">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal">And I suppose there’s no equivalent of
“timeslicing” where the cores run job A for 99% of the time
and job B, C, D, E, F, for 1% of the time.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span
style="font-size:12.0pt;color:black">From: </span></b><span
style="font-size:12.0pt;color:black">Alex Chekholko
<a class="moz-txt-link-rfc2396E" href="mailto:alex@calicolabs.com"><alex@calicolabs.com></a><br>
<b>Date: </b>Thursday, January 16, 2020 at 3:50 PM<br>
<b>To: </b>Jim Lux <a class="moz-txt-link-rfc2396E" href="mailto:james.p.lux@jpl.nasa.gov"><james.p.lux@jpl.nasa.gov></a><br>
<b>Cc: </b><a class="moz-txt-link-rfc2396E" href="mailto:beowulf@beowulf.org">"beowulf@beowulf.org"</a>
<a class="moz-txt-link-rfc2396E" href="mailto:beowulf@beowulf.org"><beowulf@beowulf.org></a><br>
<b>Subject: </b>[EXTERNAL] Re: [Beowulf] Interactive vs
batch, and schedulers<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Hey Jim,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">There is an inverse relationship between
latency and throughput. Most supercomputing centers aim to
keep their overall utilization high, so the queue always
needs to be full of jobs.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If you can have 1000 nodes always idle
and available, then your 1000 node jobs will usually take
10 seconds. But your overall utilization will be in the
low single digit percent or worse.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Alex<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Jan 16, 2020 at 3:25 PM Lux,
Jim (US 337K) via Beowulf <<a
href="mailto:beowulf@beowulf.org" moz-do-not-send="true">beowulf@beowulf.org</a>>
wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Are
there any references out there that discuss the
tradeoffs between interactive and batch scheduling
(perhaps some from the 60s and 70s?) –
<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Most
big HPC systems have a mix of giant jobs and smaller
ones managed by some process like PBS or SLURM, with
queues of various sized jobs.
<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">What
I’m interested in is the idea of jobs that, if spread
across many nodes (dozens) can complete in seconds
(<1 minute) providing essentially “interactive”
access, in the context of large jobs taking days to
complete. It’s not clear to me that the current
schedulers can actually do this – rather, they
allocate M of N nodes to a particular job pulled out
of a series of queues, and that job “owns” the nodes
until it completes. Smaller jobs get run on (M-1) of
the N nodes, and presumably complete faster, so it
works down through the queue quicker, but ultimately,
if you have a job that would take, say, 10 seconds on
1000 nodes, it’s going to take 20 minutes on 10 nodes.<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Jim<o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">-- <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
Beowulf mailing list, <a
href="mailto:Beowulf@beowulf.org" target="_blank"
moz-do-not-send="true">Beowulf@beowulf.org</a> sponsored
by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe)
visit <a
href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf"
target="_blank" moz-do-not-send="true">
https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a><o:p></o:p></p>
</blockquote>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Beowulf mailing list, <a class="moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="moz-txt-link-freetext" href="https://beowulf.org/cgi-bin/mailman/listinfo/beowulf">https://beowulf.org/cgi-bin/mailman/listinfo/beowulf</a>
</pre>
</blockquote>
</body>
</html>