[Beowulf] scheduler policy design
toon.knapen at fft.be
Thu Apr 19 07:20:15 PDT 2007
Tim Cutts wrote:
> Optimising for throughput, at least with an embarrassingly parallel
> workload of serial jobs like we have here, is trivial; a single
> first-come-first-served queue is optimal, as long as the code is well
> written, and doesn't block too much on shared resources like file
> servers or databases.
but what if you have a bi-cpu bi-core machine to which you assign 4
slots. Now one slot is being used by a process which performs heavy IO.
Suppose another process is launched that performs heavy IO. In that case
the latter process should wait until the first one is done to avoid
slowing down the efficiency of the system. Generally however, clusters
take only time and memory requirements into account.
Additionally, in the case above, for optimising the efficiency of the
node, I might prefer to launch just 1 process which uses 4 threads to
perform multi-threaded (BLAS) calculations.
> Most of the scheduling tweaks we put in to improve fairness and response
> time actually have a negative impact on throughput and efficiency.
Thanks. I totally agree but people find it hard to believe. Having a
nice article explaining all this would make it easier to explain this
(over and over again).
More information about the Beowulf