[Beowulf] Why need of a scheduler??
reuti at staff.uni-marburg.de
Thu Nov 29 07:31:56 PST 2007
Am 29.11.2007 um 14:14 schrieb amjad ali:
> I want to develop and run my parallel code (MPI based) on a Beowulf
> cluster. I have no problem as such that many user might log on to
> the cluster simultaneously. Suppose that I am free to use cluster
> dedicatedly for my single parallel application.
> 1) Do I really need a cluster scheduler installed on the cluster?
> Should I use scheduler?
if you intend to manage all by hand: no. But it will ease the usage
of the cluster for your own, as it avoids that you have to look on
your own which nodes are free.
> 2) Is there any effect/benefit on the running of a parallel code
> with or without cluster job scheduler?
It makes sense to install them even on a single machine, especially
if it has 4 cores: just to serialize your own workflow. This way you
can submit many jobs Friday evening, and all will be processed in a
controlled manner over the weekend.
> 3) How you differentiate between cluster scheduler and cluster
> resource manager?
The scheduler will define the order of jobs to be executed (by a set
up policy) and put it on nodes according to the constraints of the
> 4) If there is any significant difference between a scheduler and
> manager then plaese tell me that which of the fall in which category:
> OpenPBS, PBS Professional, SGE, Maui, Moab, Torque, Scyld, LSF,
> SLURM etc.
- Torque (as the successor of OpenPBS) is a DRM with has a FIFO
If you have parallel jobs with a varying amount of processes, the
FIFO scheduler might not be good enough to avoid starvation of jobs
with a high request for cpus as always smaller jobs slip in.
- Maui is a scheduler which needs some DRM to work for.
- SGE is a DRM which has a capable scheduler already built-in.
> 5) What is maent by " PBS/SGE/LSF supports integration with the
> Maui scheduler?
The built-in scheduler of these can be replaced by Maui (which is
only a scheduler). But in the clusters I saw which do it, I found it
always confusing to have a "qstat" command from e.g. Torque, and a
"showq" from Maui. At least for SGE I would suggest to stay with the
already built-in one.
More information about the Beowulf