[Beowulf] Prioritising Pending Jobs from different queues
dag at sonsorol.org
Mon Dec 12 17:12:47 PST 2005
The users at gridengine.sunsource.net mailing list is where quite a few
SGE guru's hang out and provide helpful suggestions.
Grid Engine is just different enough from PBS/Torque/LSF that mapping
things into clear queues sorted by priority is <ahem> an interesting
exercise. Generally speaking, the preferred behavior with SGE is that
users should request the resources that their job needs in order to
be successful and the SGE scheduler will actually sort out the most
preferred queue. The term 'queue' means different things on an LSF
system vs a SGE system, and SGE best practices gently discourage most
users from thinking of queues at all.
One possible suggestion that would get you the behavior you require
would be to use the SGE 'urgency' policy mechanism. The urgency
policy was originally put in place to allow system managers to
maximize the utilization of "expensive" assets like commercial
software licenses or monster leased supercomputers etc. Jobs
requesting the "expensive" assets would receive a boost in scheduler
preference. The amount of the boost depends on the number of urgency
tickets and the overall weight that the Urgency policy has within the
I suspect that if you enabled the Urgency policy and assigned some of
those policy entitlements to your "high priority" queue then any jobs
you have that specifically ask for the high priority queue would be
jumped to the top of the pending list for dispatch into the first
available high priority job slot.
There are many ways to do these sorts of things within SGE so no
promises that this is the "best" method. The SGE gurus would probably
ask why you even need to have 2 queues at all, especially as you seem
mostly concerned with designating some jobs as "more important" than
others which is something that can generally be done within a single
queue pretty easily using some of the other SGE policy based resource
On Dec 5, 2005, at 7:34 AM, Jenno Claesson wrote:
> Just re-asking a question for which I forgot to mention I was using
> the Sun Grid Engine.
> My question surrounds the submission of jobs into two separate queues,
> assuming I have only one host on which to run the jobs.
> If I have two queues, is there a way that I can have jobs submitted
> in one
> queue, run before jobs that were previously submitted in another
> queue. I
> am happy to wait for any currently running jobs to finish, and as
> such I
> don't really want to use subordinate_list, because I do not want to
> any running jobs. All I want to do, is ensure that the job in my
> priority" queue, is the next job to be run from the "Pending Jobs"
> I also don't want to have to use the -p option at submit time.
> all I want is a high-priority queue, and a low priority queue.
More information about the Beowulf