hyperthreading and Linux

[Greg Lindahl]

Actually, Craig, from looking at the 2.4 kernel source, it does have a
bit of hyperthreading support, enough that you could make it do the
right thing. The key item is that if it doesn't know where to run a
process, it tends to put it on an idle cpu where the associated
virtual cpu is also idle. Search for "smp_num_siblings" in kernel/sched.c,
that's 2 for a hyperthreaded cpu.

Where this goes wrong is that it will continue to put a process on the
same cpu as long as it was the last process to have run there. So if
you get 2 processes on the siblings, they'll remain there until a 3rd
process somehow gets scheduled, which is probably rare in your
environment.

To change that policy, you can hack up reschedule_idle() in sched.c. 
But you'll be slowing down the fast path.

Hm, looks to me like absolute process affinity isn't so hard to
implement... then you could hack up mpirun to set affinities.

greg