<div dir="ltr">From the cloud side of things context wise...<div><br></div><div>So I am seeing and starting to use things like <a href="http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html">http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html</a> to manage the resources. There is a lot of cool opportunities around the .socket for automation with fewer moving parts. CoreOS and some well thought out applications can be quite awesome.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 14, 2015 at 2:06 PM, Joe Landman <span dir="ltr"><<a href="mailto:landman@scalableinformatics.com" target="_blank">landman@scalableinformatics.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi folks:<br>
<br>
Its been a few years since we've had a good use case for a job scheduler, and I'll freely admit I've not paid nearly enough attention to what is currently out there.<br>
<br>
We are investigating options for a cluster/cloud scenario where I need to keep track of CPU, memory, disk used during the runs. This "keeping track" should be available via command line tools (preferably in JSON/XML/CSV output that I can easily parse).<br>
<br>
The last time we did anything in this space, I used Torque and wrote my own account summary tool: <a href="https://scalability.org/2011/03/quick-accounting-tool-for-torque/" rel="noreferrer" target="_blank">https://scalability.org/2011/03/quick-accounting-tool-for-torque/</a> , and prior to that, I did something for SGE <a href="https://arc.liv.ac.uk/pipermail/gridengine-users/2006-October/011846.html" rel="noreferrer" target="_blank">https://arc.liv.ac.uk/pipermail/gridengine-users/2006-October/011846.html</a><br>
<br>
Main requirements on the scheduler are<br>
<br>
a) a shell access. We need to be able to quickly launch a shell and limit CPU/memory usage. Cgroup control/monitoring would be terrific.<br>
<br>
b) the aforementioned accounting/usage bits. Happy to write my own data extractor (likely will need to for this project anyway) as long as I can get the data via CLI/API/...<br>
<br>
Ones I think I should be looking at include:<br>
<br>
1) SLURM<br>
2) OpenLava<br>
3) Torque<br>
<br>
What else? Has the gridengine mess ever been sorted out? And on a related note, are there any updated pages listing pro's/con's of the modern implementations of these? Again, I've not paid attention to schedulers for a while, so things may have changed a bit in a few years ...<br>
<br>
<br>
Thx!<span class="HOEnZb"><font color="#888888"><br>
<br>
-- <br>
Joseph Landman, Ph.D<br>
Founder and CEO<br>
Scalable Informatics, Inc.<br>
e: <a href="mailto:landman@scalableinformatics.com" target="_blank">landman@scalableinformatics.com</a><br>
w: <a href="http://scalableinformatics.com" rel="noreferrer" target="_blank">http://scalableinformatics.com</a><br>
t: @scalableinfo<br>
p: <a href="tel:%2B1%20734%20786%208423%20x121" value="+17347868423" target="_blank">+1 734 786 8423 x121</a><br>
c: <a href="tel:%2B1%20734%20612%204615" value="+17346124615" target="_blank">+1 734 612 4615</a><br>
_______________________________________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" rel="noreferrer" target="_blank">http://www.beowulf.org/mailman/listinfo/beowulf</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>~ Andrew "lathama" Latham <a href="mailto:lathama@gmail.com" target="_blank">lathama@gmail.com</a> <a href="http://lathama.net" target="_blank">http://lathama.net</a> ~</div></div></div>
</div>