[Beowulf] Related to: How to restrict CPU usage
akhtar_samo at yahoo.com
Mon Aug 2 13:14:45 PDT 2004
Thanx Mark & chris for flourshing my knowledge, What i want to do is that if some one executes a parallel program of 2.5 hrs execution time while this is executing(on all nodes), if i want to run a 2 minute parallel program on the same machines how will cluster (made with MPICH as an MPI) manage this.
Actually i have created a GUI environment for the execution of these parallel programs now what provision to make in that environment for the above problem............
ark Hahn <hahn at physics.mcmaster.ca> wrote:
> >>i have made a 4 node cluster now how to restrict CPU usage of a particular
> >>node i.e. if i want node 2's CPU not to exceed 70% usage.........
Chris is right. I read this very strictly (CPU meaning a particular
physical processor). but the author probably just meant "how do I configure
my queueing system to keep the load at under .7". the answer is that you
read the fine documentation of your queueing system, of course!
this is actually somewhat relevant, since anyone who talks about
loadaverage is not doing parallel-type HPC clustering. any parallel
program which has any collective operations is quite sensitive to
preemption, and so cannot tolerate competition for access to CPUs.
this sensitivity an very steep function of the size of jobs (ncpus)
and the frequency of collectives.
people who are mostly used to serial farms find this a hard concept
to grasp, somehow. we've actually thought of dynamically designating
some nodes to be parallel-only, since serial jobs can tolerate CPU
overcommittment fairly nicely.
interestingly, there are three main categories of jobs I see that would
sometimes benefit from overcommittment:
- serial jobs. they don't actually benefit, but can tolerate
overcommittment without serious degredation. and serial people
sometimes like to see that they've got jobs at least started,
even if they're not getting 100% CPU.
- gaussian jobs, even running in sysv-marallel mode. here the
issue is that gaussian is often waiting for IO to its obscenely
large scratch files. it's not unusual to see a 4-way gaussian
job *average* around 26% utilization of a 4-way node.
- interactive or other IO-intensive jobs. we have some people
who want to run interactive matlab sessions, or other programs
that have guis. mostly, they're OK if they just do this on a
head node, but that sometimes causes trouble, too.
regards, mark hahn.
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Beowulf