batch systems with job deps (afterok)

[hanzl at noel.feld.cvut.cz]

>> opensource batch system with support for job dependencies?
>> ...
>> OpenPBS (afterok feature of qsub) ... not easy to run on scyld
>> ...
>> All we need is to run 100 jobs in any order, then one job to merge
>> results, then 100 jobs again and so on.
>
>If your jobs are relatively short lived (not days long), you could do this with
>ANTS (http://unthought.net/antsd) and a Makefile to get the dependencies right.
>
>I think that could be a simple and efficient solution to your problem - but
>note that ANTS is not meant to be a real batch queue system, and it therefore
>does not have features such as queues, accounting, etc. etc.

Thanks, using 'make' this way really makes sense as our speech
recognition training is something like huge build composed of
five-minutes tasks.

What I would miss with make (and is easy with 'afterok' job
dependency) is the possibility of gradual creation of program to run -
usually I am just a few steps ahead with program debugging - previous
steps run while I prepare things which should follow (and once
prepared, I would like them to run as soon as the previous step is
finished).

But maybe it is still possible to use make, antsd and some additional
machinery, I will look into this.

Thanks

Vaclav