[Beowulf] compilers vs mpi?

[Martin Siegert]

Hi Mark,

we do exactly what you describe: compile OpenMPI with the gcc suite
and then use it with gcc, intel and open64 compilers.
This works out-of-the-box, almost.
The problem is the f90 module mpi.mod. This is (usually) a binary
file and specific to the f90 compiler that was used to compile 
OpenMPI. But there is a way to solve this problem:

1. compile openmpi using the gcc compilers, i.e., gfortran as the Fortran
   compiler and install it in /usr/local/openmpi
2. move the Fortran module to the directory
   /usr/local/openmpi/include/gfortran. In that directory
   create softlinks to the files in /usr/local/openmpi/include.
3. compile openmpi using ifort and install the Fortran module (and only
   the Fortran module!) in /usr/local/openmpi/include/intel. In that
   directory create softlinks to the files in /usr/local/openmpi/include.
4. in /usr/local/openmpi/bin create softlinks mpif90.ifort
   and mpif90.gfortran pointing to opal_wrapper. Remove the
   mpif90 softlink.
5. Move /usr/local/openmpi/share/openmpi/mpif90-wrapper-data.txt
   to /usr/local/openmpi/share/openmpi/mpif90.ifort-wrapper-data.txt.
   Change the line includedir=${includedir} to:
   includedir=${includedir}/intel
   Copy the file to
   /usr/local/openmpi/share/openmpi/mpif90.gfortran-wrapper-data.txt
   and change the line includedir=${includedir} to
   includedir=${includedir}/gfortran
6. Create a wrapper script /usr/local/openmpi/bin/mpif90:

#!/bin/bash
OMPI_WRAPPER_FC=`basename $OMPI_FC 2> /dev/null`
if [ "$OMPI_WRAPPER_FC" = 'gfortran' ]; then
   exec $0.gfortran "$@"
else
   exec $0.ifort "$@"
fi

The reason we use gfortran in step 1 is that otherwise you get those
irritating error messages from the Intel libraries, cf.
http://www.open-mpi.org/faq/?category=building#intel-compiler-wrapper-compiler-w
arnings

Cheers,
Martin

-- 
Martin Siegert
Head, Research Computing
WestGrid/ComputeCanada Site Lead
IT Services                                phone: 778 782-4691
Simon Fraser University                    fax:   778 782-4242
Burnaby, British Columbia                  email: siegert at sfu.ca
Canada  V5A 1S6

On Tue, Jul 20, 2010 at 12:07:32PM -0400, Mark Hahn wrote:
> Hi all,
> I'm interested in hearing about experiences with mixing compilers
> between the application and MPI.  that is, I would like to be able
> to compile MPI (say, OpenMPI) with gcc, and expect it to work correctly 
> with apps compiled with other compilers.  I guess I'm reasoning by analogy 
> to normal distro libs.
>
> the OpenMPI FAQ has this comment:
>
> NOTE: The Open MPI team recommends using a single compiler suite whenever
> possible. Unexpeced or undefined behavior can occur when you mix compiler
> suites in unsupported ways (e.g., mixing Fortran 77 and Fortran 90 compilers
> between different compiler suites is almost guaranteed not to work).
>
> and there are complaints elsewhere in the FAQ about f90 bindings.  I'd 
> appreciate it if someone could help a humble C/C++/perl hacker understand 
> the issues here...
>
> thanks, mark hahn.
> PS: we have a large and diverse user base, so tend to have to support gcc, 
> intel, pathscale and pgi.  we even have people who want to use intel's 
> damned synthetic 128b FP over MPI :(
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf