[Beowulf] Why need of a scheduler??

[Reuti]

Hi,

Am 29.11.2007 um 14:14 schrieb amjad ali:

> I want to develop and run my parallel code (MPI based) on a Beowulf  
> cluster. I have no problem as such that many user might log on to  
> the cluster simultaneously. Suppose that I am free to use cluster  
> dedicatedly for my single parallel application.
>
> 1) Do I really need a cluster scheduler installed on the cluster?  
> Should I use scheduler?

if you intend to manage all by hand: no. But it will ease the usage  
of the cluster for your own, as it avoids that you have to look on  
your own which nodes are free.

> 2) Is there any effect/benefit on the running of a parallel code  
> with or without cluster job scheduler?

It makes sense to install them even on a single machine, especially  
if it has 4 cores: just to serialize your own workflow. This way you  
can submit many jobs Friday evening, and all will be processed in a  
controlled manner over the weekend.

> 3) How you differentiate between cluster scheduler and cluster  
> resource manager?

The scheduler will define the order of jobs to be executed (by a set  
up policy) and put it on nodes according to the constraints of the  
resource manger.

>  4) If there is any significant difference between a scheduler and  
> manager then plaese tell me that which of the fall in which category:
>
> OpenPBS, PBS Professional, SGE, Maui, Moab, Torque, Scyld, LSF,  
> SLURM etc.

- Torque (as the successor of OpenPBS) is a DRM with has a FIFO  
scheduling strategy.

If you have parallel jobs with a varying amount of processes, the  
FIFO scheduler might not be good enough to avoid starvation of jobs  
with a high request for cpus as always smaller jobs slip in.

- Maui is a scheduler which needs some DRM to work for.

- SGE is a DRM which has a capable scheduler already built-in.

> 5) What is maent by  " PBS/SGE/LSF supports integration with the  
> Maui scheduler?

The built-in scheduler of these can be replaced by Maui (which is  
only a scheduler). But in the clusters I saw which do it, I found it  
always confusing to have a "qstat" command from e.g. Torque, and a  
"showq" from Maui. At least for SGE I would suggest to stay with the  
already built-in one.

-- Reuti