[Beowulf] Opinions of Hyper-threading?

[Ashley Pittman]

On Mon, 2008-02-25 at 14:47 -0600, Geoff Jacobs wrote:
> Bill Broadley wrote:
> > I believe it's actually simultaneous, instructions from 2 different
> > processes can run in the same cycle against 2 different register files.
> > 
> > Other chips have vertical multithreading where only 1 process runs in
> > any given cycle.
> 
> Multiple threads residing in any particular stage of execution at any
> particular time. If one thread stalls, the processor can proceed with
> execution on the other thread.
> 
> Somewhat outside the bounds of this list, but would an in order
> processor like Power 6 derive more benefit from SMT?

I saw a talk which said SMT was worth a maximum of 20% on power5 and
often performed worse than if it had been tured off.  This correlates
well with my experience of it on Intel CPUs.

http://www.hpcx.ac.uk/about/events/annual2006/slides/hague.pdf

It seems most people, myself included, benchmark(ed) with hyperthreading
disabled in the bois/at boot time and again with hyperthreading enabled
and jobs scheduled to the meta-cpu's.  Not surprisingly the performance
often isn't all that different despite having twice as many cpu's
however the variance is much higher when it's enabled.

I believe there should be a third way whereby the virtual cpu's are
enabled and running but not used to run parallel jobs, more to run any
background tasks the OS should happen to throw at them, if we were to go
down this road I could use the reclaimed cycles to do something sensible
with marshaling data for non-blocking MPI operations.  At least part of
the reason this wasn't tested before is scheduler support for
hyperthreading and CPU binding, by the time kernel support was good
enough to do the tests I'd have liked to have done the window was closed
and hardware technology had moved on.

Ashley.