[Beowulf] compilers vs mpi?

[Prentice Bisbal]

Mark Hahn wrote:
> Hi all,
> I'm interested in hearing about experiences with mixing compilers
> between the application and MPI.  that is, I would like to be able
> to compile MPI (say, OpenMPI) with gcc, and expect it to work correctly
> with apps compiled with other compilers.  I guess I'm reasoning by
> analogy to normal distro libs.
> 
> the OpenMPI FAQ has this comment:
> 
> NOTE: The Open MPI team recommends using a single compiler suite whenever
> possible. Unexpeced or undefined behavior can occur when you mix compiler
> suites in unsupported ways (e.g., mixing Fortran 77 and Fortran 90
> compilers
> between different compiler suites is almost guaranteed not to work).
> 
> and there are complaints elsewhere in the FAQ about f90 bindings.  I'd
> appreciate it if someone could help a humble C/C++/perl hacker
> understand the issues here...
> 
> thanks, mark hahn.
> PS: we have a large and diverse user base, so tend to have to support
> gcc, intel, pathscale and pgi.  we even have people who want to use
> intel's damned synthetic 128b FP over MPI :(


Mark,

I'm not a developer, but I do spend a lot of my time compiling codes
(mostly C and Fortran) for users, and I've dealt with the problem plent
of times.

Here's the library compatibility problem as I understand it:

The C programming language standard defines the syntax for symbol names
in libraries, so if you have a function named printf, when compiled into
a library, the symbol for it will always be printf.

>From what I've heard, For C++ the standard isn't as strict as C, but
usually doesn't cause any problems. I rarely compile C++ code, so I have
no real experience with this.

For Fortran, the standard doesn't define a standard naming convention
for library symbols, so the compilers have more freedom with symbol
naming. Usually, the symbol names will have 0, 1, or two underscores
before/after the symbol name, and it can be in all caps or all lower case.

This is why gfortran has these options:

-fno-underscoring
-fsecond-underscore
-fcase-lower

In theory, these options should work, but they only really work if your
using one compiler to link against libraries compiled by only one other
compilers (linking with gfortran to ifort-compiled libraries, for
example). Once you have libraries from two different compilers, the
libraries from one compiler might need --fno_underscoring, while the
other needs -fcase-lower and fsecond-underscore.

I highly recommend you read the man pages for g77 and/or gfortran where
these switches are explained. There is a much better explanation of why
they're needed there.

>From my experience, it's easier to just compile the libraries again
using a different compiler suite, and put them in a separate location to
make it clear.  For example, I have compiled Open MPI with GNU, Intel,
and PGI compilers,

$ pwd
/usr/local/openmpi

$ ls -ld *
lrwxrwxrwx 1 root root    9 Jul 17  2009 gcc -> gcc-4.1.2
drwxr-xr-x 3 root root 4096 Feb 10  2009 gcc-4.1.2
lrwxrwxrwx 1 root root    8 Jul 17  2009 intel -> intel-11
drwxr-xr-x 3 root root 4096 Feb  5  2009 intel-11
lrwxrwxrwx 1 root root    7 Jul 17  2009 pgi -> pgi-8.0
drwxr-xr-x 3 root root 4096 Jan 28  2009 pgi-8.0

As long as users specify the correct paths to include and library files
in their compiler commands, they can compile using whatever compiler
they want. To save work, I only do this for libraries that I absolutely
know that users will be using with Fortran.

-- 
Prentice