[Beowulf] multi-threading vs. MPI

[Geoff Jacobs]

Toon Knapen wrote:
> Geoff Jacobs wrote:
>>
>> If an application is going to be using only shared memory, I have no
>> doubt the consensus here is to use native threads or OpenMP. If the
>> application is going to be working over a network, MPI (or PVM) is the
>> way to go.
> 
> 
> If threads are better for SMP, but what about Numa like SGI Altix or HP
> Integrity ? On the latter for instance, inter-cell (cell = 1 board with
> 4 cpu's) memory access suffers 200 times higher latency compared to
> intra-cell memory-access.
> 
> But also the AMD (and future Intel) processors are Numa. With threads
> you have no idea about referencing remote memory but is that a good thing?
> 
> And considering that future processors are even going more extreme in
> the Numa direction (e.g. the Intel 80-core), is'nt it more future-safe
> to go with MPI if one would start a large coding-project now?
> 
> thanks for all the reactions,
> 
> toon
I do physics and computing. I am not a system programmer, so take this
with a grain of salt. However, it seems to me that a sufficiently
intelligent operating system would be able to attach the thread local
store to the local memory pool.

I've used a Superdome before (FORTRAN and HP MPI), so I know whereof you
speak, but MPI is not the general rule for software development in both
the Windows and UNIX world. Using the complexity of MPI isn't very
popular when a simpler method works okay.

However, it seems like threads have taken a little beating lately in
favor of discrete address spaces in security conscious system software
(most famously, DJBs stuff) which communicate with pipes and shm. And
lessons from QNX and BeOS tell us we might see some resurgence in the
use of message queues and other means of low level message passing in
the standard library.

-- 
Geoffrey D. Jacobs