[Beowulf] job scheduler and accounting question

[Andrew Latham]

>From the cloud side of things context wise...

So I am seeing and starting to use things like
http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
to manage the resources. There is a lot of cool opportunities around the
.socket for automation with fewer moving parts.  CoreOS and some well
thought out applications can be quite awesome.

On Tue, Jul 14, 2015 at 2:06 PM, Joe Landman <
landman at scalableinformatics.com> wrote:

> Hi folks:
>
>   Its been a few years since we've had a good use case for a job
> scheduler, and I'll freely admit I've not paid nearly enough attention to
> what is currently out there.
>
>   We are investigating options for a cluster/cloud scenario where I need
> to keep track of CPU, memory, disk used during the runs.  This "keeping
> track" should be available via command line tools (preferably in
> JSON/XML/CSV output that I can easily parse).
>
>   The last time we did anything in this space, I used Torque and wrote my
> own account summary tool:
> https://scalability.org/2011/03/quick-accounting-tool-for-torque/ , and
> prior to that, I did something for SGE
> https://arc.liv.ac.uk/pipermail/gridengine-users/2006-October/011846.html
>
>   Main requirements on the scheduler are
>
> a) a shell access.  We need to be able to quickly launch a shell and limit
> CPU/memory usage.  Cgroup control/monitoring would be terrific.
>
> b) the aforementioned accounting/usage bits.  Happy to write my own data
> extractor (likely will need to for this project anyway) as long as I can
> get the data via CLI/API/...
>
> Ones I think I should be looking at include:
>
> 1) SLURM
> 2) OpenLava
> 3) Torque
>
> What else?  Has the gridengine mess ever been sorted out?  And on a
> related note, are there any updated pages listing pro's/con's of the modern
> implementations of these?  Again, I've not paid attention to schedulers for
> a while, so things may have changed a bit in a few years ...
>
>
> Thx!
>
> --
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics, Inc.
> e: landman at scalableinformatics.com
> w: http://scalableinformatics.com
> t: @scalableinfo
> p: +1 734 786 8423 x121
> c: +1 734 612 4615
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>



-- 
~ Andrew "lathama" Latham lathama at gmail.com http://lathama.net ~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20150714/15d4676c/attachment.html>