[Beowulf] slow jobs when run through queue

[Nick Evans]

Hi all,

Under normal circumstances I would agree that the request for all 48 cores
in the machine is overkill but this particular machine has a highly
specialised FPGA card in it to do most of the heavy lifting when running a
specific set of analysis that has been tuned to run with the card. It can
only run 1 job at a time and the node itself didn't implement the normal
central software mount by design so there isn't the temptation to run
normal jobs on it and block the use of the FPGA.



Nick

On 7 Dec 2017 5:35 AM, "Tim Cutts" <tjrc at sanger.ac.uk> wrote:

Of course, if you charge for your cluster time, that hurts them in the
wallet, since they pay for all the allocated unused time.  If you don’t
charge (which is the case for us) it’s hard to incentivise them not to do
this.  Shame works, a bit.  We publish cluster analytics showing CPU
efficiency and memory efficiency league tables for the users, and that has
had some good effects in the past...

Tim

> On 6 Dec 2017, at 18:20, David Mathog <mathog at caltech.edu> wrote:
>
> on Wed, 06 Dec 2017 21:39:20 +1100 Chris Samuel wrote:
>> If this is, as I suspect is likely, bioinformatics code it could well be
that
>> it is a pipeline type application and only part of the application may
be able
>> to make use of parallelism (and then might not be very good at it).
>
> Exactly.  Super frustrating to set something like '--cpus=40' and then
watch the resulting heap of programs sit for long periods of time (hours,
not seconds) running only on a single CPU.
>
> Regards,
>
> David Mathog
> mathog at caltech.edu
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf




--
 The Wellcome Trust Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20171207/ac34a1d3/attachment.html>