[Beowulf] Please help to setup Beowulf

[Bogdan Costescu]

On Fri, 20 Feb 2009, Prentice Bisbal wrote:

> You need to take a fresh look at SGE and Open MPI.

Well, I'm subscribed to the devel lists of both these projects so I 
don't really think that I need to take a fresh look at them :-)

> Open MPI seems to be the new de facto standard MPI library

While OpenMPI would be my MPI library of choice:
- Myricom offers and supports MPICH over MX
- all IB clusters that I've had access to use MVAPICH as MPI lib of 
choice; OpenMPI is there installed only as an afterthought or because 
users ask for it; AFAIK there is no preference for OpenMPI in OFED
- there is no support for Quadrics cards in OpenMPI (unless they are 
used as 10GE cards)

So maybe you'd like to explain your choice of words regarding 
OpenMPI's usage...

> and you can compile it to be fully integrated with both SGE and 
> Torque

Full integration means different things for the 2 batch systems...

LAM/MPI 7.x works with SGE mainly because of my efforts at that time; 
after that point, one could also claim that LAM/MPI had full 
integration with SGE. This still meant however that a rsh (SGE's own 
rsh based on NetBSD one) was used to start processes on remote nodes.

> I just set up a cluster using SGE and Open MPI (built with the 
> --with-sge option), and I there's no need to tinker with the SGE's 
> MPI startup wrapper scripts like in the past.

Well, Sun is involved a lot in the OpenMPI development and it makes a 
lot of sense for them to have as good integration as possible with 
their own offering. This doesn't restrict to only SGE but also f.e. to 
the ability to compile OpenMPI with Sun's compilers. (I have no ties 
with Sun :-)).

> I could be wrong but I think at that time, to use Torque you needed 
> to compile a separate mpiexec program developed by a 3rd party to 
> get "tight integration" between MPI and Torque.

If you use MPICH1 based libraries (like the above mentioned one from 
Myricom), you need to use OSC's mpiexec even today.

-- 
Bogdan Costescu

IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
Phone: +49 6221 54 8240, Fax: +49 6221 54 8850
E-mail: bogdan.costescu at iwr.uni-heidelberg.de