Processor contention(?) and network bandwidth on AMD

[Velocet]

On Mon, Apr 29, 2002 at 03:40:30PM -0400, Mark Hahn's all...

> and of course, there's a big update to the scheduler impending or 
> already merged ("Ingo's O(1) scheduler").  it seems to be a lot smarter
> about issues like migrating procs, affinity, waking up the right tasks, etc.
> 
> > unloaded:                                         11486.6 KB/real sec
> > 2 matlab simulations:                             10637.8 KB/real sec
> > 2 matlab simulations and 2 SETI at homes (nice -19):  6645.4 KB/real sec
> 
> SETI at home is obviously in the "so don't do that" category.  I expect your
> matlab was decelerated by a similar amount.
> 
> though I think that another of Ingo's goals with the new scheduler
> was to give more intuitive nice -19 behavior.  that is, most people 
> think of nice -19 as a way to spend otherwise idle CPU on something.
> Linux (and at least some other Unixes) have *NEVER* done this - 
> figure around 5% CPU, and that's ignoring cache/etc effects.
> anyway, I think Ingo tries to keep -19 pretty close to idle-only.
> (there are fundamental issues with really implementing an idle-only
> form of scheduling, since you wind up with "priority inversion",
> where the idle-only task holds a lock when high-pri jobs want to do
> something...)

How does freebsd do this then? They've had idle (and realtime) priority
in the kernels for a couple years. And there are no problems with
priority inversion (which was Mike Shaver's answer to me for linux's
lack of idle time priority 2 years ago when I asked him if Linux was
going to be incorporating it) - rather, they have lock breaking code working
nicely in freebsd.

Freebsd's idle priority gives 30 levels of idle priority to play with.
Anything at a lower level of idle priority gets NO time on the cpu at all
until there is some available. This is quite nice when things like Gaussian98
is running and you want to put a higher priority g98 job on without having
a nice level 19 in linux G98 fighting and thrashing the cache vs the
nice level 0 g98. I notice a total speedup of 1-2% at least in the difference
between running the two jobs sequentially vs putting 1 at 19 and 1 at 0.

I find this system VERY useful for scheduling jobs on various machines
shared by different groups. It really guarantees that the CPU will be
used primarily for one type of job and not another and avoids
cache thrashing quite nicely - until something goes to disk, and then
the idle job wakes up, etc... is it worth using the extra cpu and possibly
thrashing the cache, or is it more efficient to wait for a bigger chunk
of free CPU?

/kc


> 
> > Ouch.  This is on RedHat 7.2 with kernel 2.4.9-31.
> 
> ugh.
> 
> NOTE TO ALL BEOWULF USERS: seriously consider running 2.4.18
> or better, and *definitely* try out gcc 3.1 ASAP.  
> 
> these updates have changed my life </evangelical>
> 
> 
> > 2.4.18 kernel (the one from SGI's 1.1 XFS release).  All showed the same 
> > results (well, 2.4.18 didn't show much of a drop with just the two 
> > matlabs, but still crashed with matlab+SETI).  The one Intel system I 
> 
> hmm, come to think of it, I think Ingo's scheduler isn't merged 
> in 2.4.19-pre yet.
> 
> > tested (dual PIII 933 on an i860) showed very little bandwidth drop with 
> 
> i860 is a P4 chipset afaik...
> 
> > load, and no extra drop for an overload.
> 
> then again, it would not be astonishing if PIII and P4 showed
> dramatically different effects of cache pollution.  remember that 
> the P4 depends rather strongly on seeing a decent hit rate in 
> its many caches (trace cache, normal I+D caches, TLB, prediction tables, etc)
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Ken Chase, math at velocet.ca  *  Velocet Communications Inc.  *  Toronto, CANADA