[Beowulf] slow jobs when run through queue

[Nick Evans]

I completely agree. We have a web page where people can see

   - where their jobs are running
   - what sort of resources were requested
   - the peak resources actually used
   - wall time remaining (orange highlighted at 20% remaining and red at
   10% remaining)


On 20 December 2017 at 03:41, Peter Clapham <pc7 at sanger.ac.uk> wrote:

> Show back of utilization and use patterns openly also removes admins from
> being “the Police”.
>
> Instead each user of the system can see who is requesting excessive
> memory, using inappropriate queues or just inefficient workloads at scale.
> This creates a self-Policing environment and certainly both re-enforces a
> community feel and improves communication between the groups of users.
> Pete
>
> On 12/6/17, 6:36 PM, "Beowulf on behalf of Tim Cutts" <
> beowulf-bounces at beowulf.org on behalf of tjrc at sanger.ac.uk> wrote:
>
>     Of course, if you charge for your cluster time, that hurts them in the
> wallet, since they pay for all the allocated unused time.  If you don’t
> charge (which is the case for us) it’s hard to incentivise them not to do
> this.  Shame works, a bit.  We publish cluster analytics showing CPU
> efficiency and memory efficiency league tables for the users, and that has
> had some good effects in the past...
>
>     Tim
>
>     > On 6 Dec 2017, at 18:20, David Mathog <mathog at caltech.edu> wrote:
>     >
>     > on Wed, 06 Dec 2017 21:39:20 +1100 Chris Samuel wrote:
>     >> If this is, as I suspect is likely, bioinformatics code it could
> well be that
>     >> it is a pipeline type application and only part of the application
> may be able
>     >> to make use of parallelism (and then might not be very good at it).
>     >
>     > Exactly.  Super frustrating to set something like '--cpus=40' and
> then watch the resulting heap of programs sit for long periods of time
> (hours, not seconds) running only on a single CPU.
>     >
>     > Regards,
>     >
>     > David Mathog
>     > mathog at caltech.edu
>     > Manager, Sequence Analysis Facility, Biology Division, Caltech
>     > _______________________________________________
>     > Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
>     > To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>
>
>     --
>      The Wellcome Trust Sanger Institute is operated by Genome Research
>      Limited, a charity registered in England with number 1021457 and a
>      company registered in England with number 2742969, whose registered
>      office is 215 Euston Road, London, NW1 2BE.
>     _______________________________________________
>     Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> Computing
>     To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
>
>
>
>
> --
>  The Wellcome Trust Sanger Institute is operated by Genome Research
>  Limited, a charity registered in England with number 1021457 and a
>  company registered in England with number 2742969, whose registered
>  office is 215 Euston Road, London, NW1 2BE.
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20171220/7a834768/attachment.html>