[Beowulf] Shared memory

[Mark Westwood]

Jeff

We too have a Beowulf cluster with nodes having 2 CPUs. The memory in 
each node is shared between the 2 CPUs. We use Grid Engine to place 
processes onto processors at run time.  Some of our codes require a lot 
of memory for each process, and will be placed one process on a node, 
some of our codes require less memory and will be placed two processes 
to a node.  There are different queues defined for the two types of job.

Our MPI codes are 'processor / node blind' in the sense that there are 
no special features of the code arising directly from the possibility 
that, at run time, two processes executing the program might be sharing 
a node.  Users determine, prior to a job's execution, how much memory 
each process in a parallel computation will require, and submit the job 
to the right queue.

My thinking is that mixed-mode programming, in which a code uses MPI for 
inter-node communication and shared-memory programming (eg OpenMP) for 
intra-node communication, is not worth the effort when you only have 2 
CPUs in each node.  In fact, my thinking is that it's not even worth the 
time experimenting to gather evidence.  I guess that makes me prejudiced 
against mixed-mode programming on a typical COTS cluster.  Now, if you 
were to offer me, say, 8 or 16 CPUs per node, I might think again.  Or 
indeed if I am shot down in flames by contributors to this list who have 
done the experiments ...


One reference which might be of interest:

http://www.epcc.ed.ac.uk/DIRECT/newsletter9/mixed.html

Regards
Mark Westwood


Jeff zhang wrote:
> Each node of my Beowulf cluster has two CPUs. The memory is shared
> between the two CPUs. How is MPI handling the memory in this
> situation? What is the most efficient way to program under this
> situation?
> Thanks.
> 
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
> 
> 
> 

-- 
Mark Westwood
Parallel Programmer
OHM Ltd
The Technology Centre
Offshore Technology Park
Claymore Drive
Aberdeen
AB23 8GD
United Kingdom

+44 (0)870 429 6586
www.ohmsurveys.com

-------------------------------------------------------------------------------
As usual in such postings, the views expressed in this email are 
personal and are not intended to represent the views of OHM Ltd.
-------------------------------------------------------------------------------