Compiling Beowulf software

[Patrick Geoffray]

On Fri, 8 Dec 2000, Josip Loncaric wrote:

> memory performance using usysv transport on our SMP boxes is about as
> good as the hardware can deliver (1 microsecond latency, 266 Mbyte/s
> peak bandwidth).  MPICH shared memory performance is not as good (16
> microsecond latency, 235 Mbyte/s peak bandwidth).  On the minus side,

Josip,

I am very surprised by the SMP performance. 1 us is very very (too) low,
it's the cost a of a system call. usysv uses SYS V semaphores, and I don't
think it's possible to reach this level of latency with them.

The bandwidth is also surprising, I suspect a cache effect in your test.
Which processor do you use ? Special memory ? The bandwidth of MPICH
SMP is, on my testbed, more around 100 MB/s (using 2 memory copies).

The best I was able to get with a very optimized SMP implementation is 2
us on very fast processors without any locks and 250 MB/s using a direct
memory copy between processes memory spaces.

Regards.

Patrick Geoffray