[Beowulf] Related to: How to restrict CPU usage

[akhtar Rasool]

Hi,
Thanx Mark & chris for flourshing my knowledge, What i want to do is that if some one executes a parallel program of 2.5 hrs execution time while this is executing(on all nodes), if i want to run a 2 minute parallel program on the same machines how will cluster (made with MPICH as an MPI) manage this.
Actually i have created a GUI environment for the execution of these parallel programs now what provision to make in that environment for the above problem............
 
Akhtar

ark Hahn <hahn at physics.mcmaster.ca> wrote:
> >>i have made a 4 node cluster now how to restrict CPU usage of a particular
> >>node i.e. if i want node 2's CPU not to exceed 70% usage......... 

Chris is right. I read this very strictly (CPU meaning a particular 
physical processor). but the author probably just meant "how do I configure
my queueing system to keep the load at under .7". the answer is that you 
read the fine documentation of your queueing system, of course!

this is actually somewhat relevant, since anyone who talks about
loadaverage is not doing parallel-type HPC clustering. any parallel 
program which has any collective operations is quite sensitive to 
preemption, and so cannot tolerate competition for access to CPUs.
this sensitivity an very steep function of the size of jobs (ncpus)
and the frequency of collectives.

people who are mostly used to serial farms find this a hard concept
to grasp, somehow. we've actually thought of dynamically designating
some nodes to be parallel-only, since serial jobs can tolerate CPU
overcommittment fairly nicely.

interestingly, there are three main categories of jobs I see that would 
sometimes benefit from overcommittment:
- serial jobs. they don't actually benefit, but can tolerate 
overcommittment without serious degredation. and serial people
sometimes like to see that they've got jobs at least started, 
even if they're not getting 100% CPU.

- gaussian jobs, even running in sysv-marallel mode. here the 
issue is that gaussian is often waiting for IO to its obscenely
large scratch files. it's not unusual to see a 4-way gaussian
job *average* around 26% utilization of a 4-way node.

- interactive or other IO-intensive jobs. we have some people 
who want to run interactive matlab sessions, or other programs
that have guis. mostly, they're OK if they just do this on a
head node, but that sometimes causes trouble, too.

regards, mark hahn.

_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

		
---------------------------------
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.beowulf.org/pipermail/beowulf/attachments/20040802/598bcd38/attachment.html>