[Beowulf] Re: Beowulf Digest, Vol 17, Issue 17

[Don Kinghorn]

Hi Mark,

Yes, for the most part the bios settings on the 2875 don't have much impact on 
performance.  [It was an inexpensive ATX form factor board which I needed at 
the time and real performance with our code was pretty good] The 2891 looks 
to be a MUCH better board (and gives marginally better performance with our 
code as is expected).  

I had to set both interleaving settings to "Auto" on the 2875 to get it to 
play nice with 4GB memory when the AMD 248 Opterons changed from "model 5 
stepping 10" to "model 37 stepping 0"   It was just a painfull trial and 
error process to get to things to work ... no performance impact ... no real 
senseable reason.

"you do that voodoo so well"  :-)

-Don    

> > >there are only 4 slots on the Tyan 2875 (I had mistakenly reported
> > > yesterday
> >
> > I'm not seeing anywhere at Tyan an indication this board can take
> > advantage of NUMA.
>
> node interleave is meaningless for the 2875, since the board only has
> memory attached to one CPU.  while the bios probably does include the
> ACPI table that informs the kernel's k8-numa code, it's moot, since
> there's no way to arrange cpu-proc affinity to minimize non-local
> accesses.  (except by not using the second socket, of course!)
>
> I'd expect NUMA support to make more of a difference on 4-socket systems,
> since on them, a process can be >1 hop away from memory.  on a 2-socket
> system, it's probably still worth doing, but can't be all that critical.
>
> naturally, latency-sensitive codes (big but with poor locality) will
> show a bigger difference.
>
> > >Bank interleaving "Auto"
>
> I tried to measure this on a dual, and couldn't.  it's hard to see,
> based on the low-level hardware specs, why it would matter much.
> yes, bank interleave should reduce the amount of time waiting on
> bank misses, but it's certainly not visible to Stream.
>
> > >Node interleaving "Auto"
>
> turning this to on essentially defeats NUMA; it could be the right thing
> for some codes/systems, since it means that no process has any special
> affinity for a particular socket.

-- 
Dr. Donald B. Kinghorn Parallel Quantum Solutions LLC
http://www.pqs-chem.com