[Beowulf] cluster scheduler for dynamic tree-structured jobs?

[Guy Coates]

On 15/05/10 11:24, Andrew Piskorski wrote:
> Folks, I could use some advice on which cluster job scheduler (batch
> queuing system) would be most appropriate for my particular needs.
> I've looked through docs for SGE, Slurm, etc., but without first-hand
> experience with each one it's not at all clear to me which I should
> choose...
> 

This may be late in the day but...
If you job dependencies are too complicated for you queuing system to
deal with, you may want to look at the Ensembl Hive system;

http://www.ensembl.org/info/docs/eHive/index.html

It is the system we use in-house for our genome-analysis pipelines,
which have lots of complicated dependencies. It sits on top of a
traditional queuing system which handles job-dispatch etc.

It has been de-coupled from the genome analysis workflow, so you should
(in theory) be able to use it for any analysis.

Cheers,

Guy

-- 
Dr. Guy Coates, Informatics System Group
The Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1HH, UK
Tel: +44 (0)1223 834244 x 6925
Fax: +44 (0)1223 496802


-- 
 The Wellcome Trust Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE.