[Beowulf] PGI_POST_COMPILE_OPTS

[Gus Correa]

Hi Brock, Alan, list

First off ...

Brock: thanks for the RCE Podcast.
It is great!
For those who don't know it,
and are interested in all aspects of HPC,
here is the link:

http://www.rce-cast.com/

Brock Palen wrote:
 > I noticed that the cray XT machines are setting an environment variable
 > 'PGI_POST_COMPILE_OPTS'  to -tp barcelona-64
 >
 > Is this a cray specific thing?

It seems to be a Cray thing indeed.
There are equivalent flags for GNU and PATHSCALE on Cray also.
See these links:

http://www.nccs.gov/computing-resources/jaguar/software/?&software=libsci
http://www.nics.tennessee.edu/user-support/software?&software=libsci

See also pages 65-67 of this Cray document:

http://docs.cray.com/books/S-2393-13/S-2393-13.pdf

I would guess the Cray "libsci" somehow adds those flags on the fly
when you launch the PGI compilers (or alias the pgi compiler commands).

On our (non-Cray) Linux cluster with
AMD quad-core I have to include those optimization flags by hand.
In our case, for AMD Shanghai it is -tp shanghai-64.
Likewise for Intel machines (with different -tp flags, of course).

I have checked a number of PGI files and there is no such a thing
as a PGI_POST_COMPILE_OPTS environment variable.
For instance, their suggestions for "module" files set these environment
variables:
PGI, CC, CPP, CXX, FC, F77, F90, PATH, MANPATH, and LD_LIBRARY_PATH.
However, no machine-dependent optimization flag is set,
no PGI_POST_COMPILE_OPTS either.
(The default machine-independent optimization level,
according to man pgf90, is -01.)

You can check these module.skel files in:
pgi/8.0-4/linux86-64/8.0-4/etc/modulefiles/ (whatever version you have
mine is 8.0-4).

Note also that the PGI directory tree has a specific "cray" directory, 
with several libraries, which suggests that PGI does some
special voodoo on Cray machines.

Intel has the ifort.cfg, icc.cfg, icpc.cfg files that allow the
compilers to be configured/customized with user/system options.
I couldn't find an equivalent scheme on PGI.

My $0.02.
Gus Correa

---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------

 > the -tp flag is a normal pgi compiler
 > option, I want to know if I can use this variable to set options for
 > specific people (module files),  and not make a change global in localrc.
 >
 > I can't find it documented any place, other than the cray module files,
 > and I can't find its value any place looking on NICS Kraken,  any input
 > would be great to know if I can use this option or not.
 >
 >
 > Brock Palen
 > www.umich.edu/~brockp
 > Center for Advanced Computing
 > brockp at umich.edu
 > (734)936-1985
 >
 >
 >

Alan Louis Scheinine wrote:
> Brock Palen,
> 
> In your original posting you spoke of setting options using 
> PGI_POST_COMPILE_OPTS
> without specifying the type of option.  In your follow-up post you wrote
> 
>> using PGI's unified binary, to support all of these
>> -tp x64,amd64e,barcelona-64
> 
> Interesting.  A PGI WWW page says:
> PGI compilers can generate a single PGI Unified Binary™ executable
>  > fully optimized for both Intel EM64T and AMD64 processors, delivering
>  > all the benefits of a single x64 platform while enabling you to leverage
>  > the latest innovations from both Intel and AMD.
> 
> For different AMD revisions also in the same binary?  Reading the 
> manual, it seems
> that there are no restrictions on what can be in the "-tp" list.
> 
> The environment variable PGI_POST_COMPILE_OPTS seems like the appropriate
> place to set the "-tp" options.  Tell use whether the Cray digests this
> innovative use.
> 
> Alan
>