[Beowulf] multi-threading vs. MPI

[Eray Ozkural]

On Dec 11, 2007 2:40 AM, Joe Landman <landman at scalableinformatics.com> wrote:
> Our view has always been use what you are comfortable with, and what you
> need.  If you need to run across a cluster, use MPI.  If you need to run
> across a single large memory machine, use OpenMP.
>
> FWIW:  I would suggest learning both.  With the advent of many-core
> workstations, and accelerator systems with many many cores, programming
> these things is more likely to be mediated by a compiler (OpenMP like)
> than putting MPI stacks on the Cell SPUs (not enough local scratchpad
> ram for it).
>
> Just my $0.02, and I hope I generated light, and very little heat.

Thanks for your comments. I've seen that OpenMP is much easier to
develop with than MPI, which in my experience takes a lot of time to
program with due to its low-level nature and complicated side-effects.
I would in fact prefer to use implicit parallelism in a high-level
language (functional) to program complicated memory architectures.
Which doesn't exist in the way I imagine it due to the
short-sightedness of programming language people. I don't think that
the programmer should be too involved with the exact details of
caches, etc. It doesn't make too much sense to me. Back in the day, I
learned how to write assembly code that fits in some 256 byte code
cache. But how do you do that kind of optimization in a large scale
NUMA architecture, what if you move the code to a slightly different
architecture? The OpenMP model is superior to old ways of programming
shared memory systems (like the awful pthreads), but as the
architectures get more complicated I doubt that it will allow the
programmers to extract sufficient performance from those systems.

BTW, is OpenMP usable with the Cell processor on the Playstation 3? I wondered.

Best,

-- 
Eray Ozkural, PhD candidate.  Comp. Sci. Dept., Bilkent University, Ankara
http://www.cs.bilkent.edu.tr/~erayo  Malfunct: http://myspace.com/malfunct
ai-philosophy: http://groups.yahoo.com/group/ai-philosophy