[Beowulf] scheduler policy design

[Toon Knapen]

Tim Cutts wrote:
> Optimising for throughput, at least with an embarrassingly parallel 
> workload of serial jobs like we have here, is trivial; a single 
> first-come-first-served queue is optimal, as long as the code is well 
> written, and doesn't block too much on shared resources like file 
> servers or databases.


but what if you have a bi-cpu bi-core machine to which you assign 4 
slots. Now one slot is being used by a process which performs heavy IO. 
Suppose another process is launched that performs heavy IO. In that case 
the latter process should wait until the first one is done to avoid 
slowing down the efficiency of the system. Generally however, clusters 
take only time and memory requirements into account.

Additionally, in the case above, for optimising the efficiency of the 
node, I might prefer to launch just 1 process which uses 4 threads to 
perform multi-threaded (BLAS) calculations.


> Most of the scheduling tweaks we put in to improve fairness and response 
> time actually have a negative impact on throughput and efficiency.

Thanks. I totally agree but people find it hard to believe. Having a 
nice article explaining all this would make it easier to explain this 
(over and over again).

t