[Beowulf] scheduler policy design
tjrc at sanger.ac.uk
Fri Apr 20 00:56:11 PDT 2007
On 20 Apr 2007, at 8:10 am, John Hearns wrote:
> Toon Knapen wrote:
>> Additionally, in the case above, for optimising the efficiency of
>> the node, I might prefer to launch just 1 process which uses 4
>> threads to perform multi-threaded (BLAS) calculations.
> We help out customers deal with problems like that, and the above
> example on serial jobs.
> The answer to this one would be to create a parallel queue with
> just one job slot, and prevent any other jobs from running on the
> node. The parallel job then gets the full resources of the node,
> and can run the multi-threaded calculations you want.
> If you don't want jobs to be allocated that way, we have a
> 'multiway' queue for running several jobs per host.
But remember you don't necessarily need to have separate queues if
your scheduler supports asking for multiple job slots on a single
machine (as in the LSF example I gave in my last posting). John's
suggestion is an alternative way to achieve the same thing, although
it introduces other questions, such as what's the relative priority
of the two queues, and does the scheduler support backfill scheduling
between the queues to help short single threaded jobs run while it is
reserving processors for the parallel job?
More information about the Beowulf