how to run gamess parallel on beowulf cluster?

Art Edwards edwardsa at plk.af.mil
Wed Sep 25 09:09:50 PDT 2002


I have chosen not to use ddikick. Rather, I compile using g77 with
the appropriate include files. So to accomplish this requires a few
hacks in the gamess code. One is to allow full paths for files 
(As I recall you have to do this in iolib.src). Also, you will have to
put some sort of sleep routine somewhere. As I look at our sources I
don't find it. Perhaps it is in the MPI libraries.

I will attach my compall, comp and lked scripts as well. I based these
on other MPI builds. If you are running on the current (bz8) scyld
then some of what we had to do about getting things to run on the
internal nodes is probably irrelevant, so I will not include them.

Art Edwards

Senior Research Physicist
Air Force Research Laboratory

P. S. Note that the scripts use g77-3.2. This can be replace with the
standard g77. let me know how this goes.

On Wed, Sep 25, 2002 at 09:54:34AM -0500, zhao gump wrote:
> Thanks a lot!
> yes, I am trying to run gamess ib scyld distribution beowulf cluster.
> what I did so far is:
> yes, I can compile source code, with g77 or portland group pgf77.
> I can run each job on any single cpu.
> but when I try to run single job on multiple cpus, maybe so called
> running parallelly, can't accomplish~
> 
> I know gamess use ddikick to distribute tasks to differrent cpus and
> it use "rsh". but scyld beowulf use bpsh. I tried to alias rsh=bpsh
> or change ddikick.c source code but either way doesn't work.
> 
> so, could you please tell me how you did to run single job on several
> CPUs? if you don't mind, I would like to know any details. if you can
> email me the running script too, that will be sooo great.
> 
> thanks thanks thanks a lot. you are my last straw.
> 
> Have a wonderful day!
> 
> 
> >From: Art Edwards <edwardsa at plk.af.mil>
> >To: zhao gump <gumpzhao at hotmail.com>
> >CC: beowulf at beowulf.org
> >Subject: Re: how to run gamess parallel on beowulf cluster?
> >Date: Tue, 24 Sep 2002 16:41:09 -0600
> >
> >I am running gamess using the scyld distribution. If this is
> >of interest let me know and I will tell you what I did.
> >
> >Art Edwards
> >On Tue, Sep 24, 2002 at 11:37:07AM -0500, zhao gump wrote:
> >> Dear all,
> >>  I am totally a blind rookie for beowulf and I am trying to
> >> run gamess on our new built 16 nodes beowulf cluster.
> >>
> >> There is no "rsh" command in this scyld beowulf. I tried to
> >> change "rsh" in ddikick.c to "bpsh" or use "alias" to make
> >> bpsh equal to rsh. both of them don't work.
> >> It can run without any problem on any single cpu.
> >>
> >> I am desperately hoping someone can tell me how to get this done.
> >>
> >> Thanks a million times!!!
> >>
> >> ==============================
> >>
> >>
> >>
> >> _________________________________________________________________
> >> Send and receive Hotmail on your mobile device: http://mobile.msn.com
> >>
> >> _______________________________________________
> >> Beowulf mailing list, Beowulf at beowulf.org
> >> To change your subscription (digest mode or unsubscribe) visit
> >> http://www.beowulf.org/mailman/listinfo/beowulf
> 
> 
> 
> 
> _________________________________________________________________
> MSN Photos is the easiest way to share and print your photos: 
> http://photos.msn.com/support/worldwide.aspx
> 
-------------- next part --------------
#!/bin/csh
#
#   9 Nov 01 - MWS - C-shell script to compile all of GAMESS.
#
#  Invoke this script by 'compall >& compall.log &'.
#  UNIX compilers are often real slow, so start this up, and go for coffee.
#
#  ----- The next two lines require your attention.
#  Choose your machine, the legal values for TARGET are
#      compaq-axp,compaq-shmem,convex-spp,cray-pvp,cray-sv1,cray-t3e,
#      fuji-vpp,fuji-ap,hitachi,hp9000,ibm32,ibm64,ibm-sp,linux-pc,
#      macosx,necsx,sgi-mpi,sgi,sun
#  Choose your GAMESS directory tree.
#
set TARGET=linux-pc-mpi
chdir /home/edwardsa/gamess-current/gamess
#
#  ----- and now the script begins...
#
date
#
#  Check to see if the installer has built actvte.x
#
if (!(-x tools/actvte.x)) then
   echo You did not compile the source code activator actvte.x
   echo Please read and follow the directions in $cwd/tools/readme.unix
   exit 8
endif
#
#  Compile a little bit of C to allow for runtime memory allocation.
#  For some systems, there is additional code in this file...
#
set extraflags=' '
if ($TARGET == compaq-axp)   set extraflags='-DCOMPAQ'
if ($TARGET == compaq-shmem) set extraflags='-DCOMPAQ'
if ($TARGET == convex-spp)   set extraflags='-DHP9000'
if ($TARGET == cray-pvp)     set extraflags='-DCRAY'
if ($TARGET == cray-sv1)     set extraflags='-DCRAY'
if ($TARGET == cray-t3e)     set extraflags='-DCRAY'
if ($TARGET == fuji-app)     set extraflags='-DFUJITSU'
if ($TARGET == fuji-vp)      set extraflags='-DFUJITSU'
if ($TARGET == hitachi)      set extraflags='-DHP9000'
if ($TARGET == ibm32)        set extraflags='-DIBM32'
if ($TARGET == ibm64)        set extraflags='-q64 -DIBM64'
if ($TARGET == ibm-sp)       set extraflags='-DIBMSP'
if ($TARGET == hp9000)       set extraflags='-DHP9000'
if ($TARGET == linux-pc)     set extraflags='-DLINUX'
if ($TARGET == linux-pc-mpi)     set extraflags='-DLINUX'
if ($TARGET == macosx)       set extraflags='-DLINUX'
if ($TARGET == necsx)        set extraflags='-DNECSX'
if ($TARGET == sgi-mpi)      set extraflags='-64 -mips4 -DSGI'
if ($TARGET == sgi)          set extraflags='-mips2 -DSGI'
if ($TARGET == sun)          set extraflags='-DSUN'
set echo
gcc-3.2 -c $extraflags source/zunix.c
unset echo
mv zunix.o object/zunix.o
#
#    GAMESS uses three interfaces to message passing libraries,
#    so now we must pick the right one.
#
switch ($TARGET)
#               SHMEM interface, there's no C to compile
   case compaq-shmem:
   case cray-t3e:
      breaksw
#               MPI-1 interface, there's no C to compile
   case cray-sv1:
   case ibm-sp:
   case necsx:
   case sgi-mpi:
   case linux-pc-mpi
      breaksw
#               TCP/IP sockets interface
#        this case has two C language files to compile
   case compaq-axp:
   case hp9000:
   case ibm32:
   case ibm64:
   case linux-pc:
   case macosx:
   case sgi:
   case sun:
      if ($TARGET == compaq-axp) set socketflags='-DCOMPAQ'
      if ($TARGET == cray-sv1)   set socketflags='-DCRAY'
      if ($TARGET == hp9000)     set socketflags='-DHP9000'
      if ($TARGET == ibm32)      set socketflags='-DIBM32'
      if ($TARGET == ibm64)      set socketflags='-q64 -DIBM64'
      if ($TARGET == linux-pc)   set socketflags='-DLINUX'
      if ($TARGET == macosx)     set socketflags='-DLINUX'
      if ($TARGET == sgi)        set socketflags='-DSGI -mips2'
      if ($TARGET == sun)        set socketflags='-DSUN'
      set socketlibs=' '
      if ($TARGET == sun) set socketlibs='-lsocket -lnsl'
      set echo
      cc -o ddikick.x $socketflags source/ddikick.c $socketlibs
      cc -c $socketflags source/ddisoc.c
      unset echo
      mv ddisoc.o object/ddisoc.o
      breaksw
#               Compile for sequential execution only, so there's no C.
   case cray-pvp:
      breaksw
#               no interface currently available,
#               but it is likely that all of these can use MPI-1...
   case convex-spp:
   case fuji-vpp:
   case fuji-ap:
   case hitachi:
      echo This machine does not yet have modernized message passing code.
      echo Please see REFS.DOC about what needs to be done to port to your
      echo machine type.  The compiling process aborts...
      exit 4
#               and this must be a typo
   default:
      echo Unrecognized TARGET $TARGET selected in compall.
      exit 8
endsw
#
#  Compile the FORTRAN part of DDI.
#  Both MPI and socket interfaces are included in DDI.
#  The SHMEM interface is in DDISHM.
#
switch ($TARGET)
   case compaq-shmem:
   case cray-t3e:
      ./comp ddishm
      breaksw
   default:
      ./comp ddi
endsw
#
#  If we do not have the BLAS in a system library,
#  compile a FORTRAN version of these supplied with GAMESS
#
if ($TARGET == compaq-axp) ./comp blas
if ($TARGET == fuji-ap)    ./comp blas
if ($TARGET == linux-pc)   ./comp blas
if ($TARGET == macosx)     ./comp blas
if ($TARGET == sun)        ./comp blas
#
if ($TARGET == ibm64)      ./comp blaswrap
#
#  If this is a vector machine, we must compile the vector routines
#
if ($TARGET == cray-pvp) ./comp vector
if ($TARGET == cray-sv1) ./comp vector
if ($TARGET == fuji-vpp) ./comp vector
if ($TARGET == necsx)    ./comp vector
#
#  ---------- activate and compile the individual source files -----------
#
./comp aldeci
./comp algnci
./comp basecp
./comp basext
./comp bashuz
./comp bashz2
./comp basn21
./comp basn31
./comp bassto
./comp chgpen
./comp cphf
./comp cprohf
./comp cosmo
./comp delocl
./comp dft
./comp dftaux
./comp dftexc
./comp dftfun
./comp dftgrd
./comp dftint
./comp dmulti
./comp drc
./comp ecp
./comp ecpder
./comp ecphw
./comp ecplib
./comp ecpsbk
./comp efdrvr
./comp efelec
./comp efgrd2
./comp efgrda
./comp efgrdb
./comp efgrdc
./comp efinp
./comp efinta
./comp efintb
./comp efpaul
./comp efpcm
./comp efpcov
./comp eigen
./comp ffield
./comp frfmt
./comp gamess
./comp globop
./comp gradex
./comp grd1
./comp grd2a
./comp grd2b
./comp grd2c
./comp guess
./comp gugdga
./comp gugdgb
./comp gugdm
./comp gugdm2
./comp gugdrt
./comp gugem
./comp gugsrt
./comp gvb
./comp hess
./comp hss1a
./comp hss1b
./comp hss2a
./comp hss2b
./comp inputa
./comp inputb
./comp inputc
./comp int1
./comp int2a
./comp int2b
./comp iolib
./comp lagran
./comp local
./comp loccd
./comp locpol
./comp mccas
./comp mcjac
./comp mcqdpt
./comp mcqdwt
./comp mcqud
./comp mcscf
./comp mctwo
./comp morokm
./comp mp2
./comp mp2ddi
./comp mp2grd
./comp mpcdat
./comp mpcgrd
./comp mpcint
./comp mpcmol
./comp mpcmsc
./comp mthlib
./comp nameio
./comp ordint
./comp parley
./comp pcm
./comp pcmcav
./comp pcmder
./comp pcmdis
./comp pcmief
./comp pcmpol
./comp pcmvch
./comp prpel
./comp prplib
./comp prppop
./comp qfmm
./comp qmfm
./comp qmmm
./comp raman
./comp resc
./comp rhfuhf
./comp rxncrd
./comp ryspol
./comp scflib
./comp scfmi
./comp scrf
./comp sobrt
./comp soffac
./comp solib
./comp sozeff
./comp statpt
./comp surf
./comp symorb
./comp symslc
./comp tdhf
./comp trans
./comp trfdm2
./comp trnstn
./comp trudge
./comp unport
./comp vibanl
./comp vscf
./comp zheev
./comp zmatrx
#
#      optional SIMOMM method using Tinker MM code
set TINKER=false
if ($TINKER == true) then
   ./comp Libtad
   ./comp Libteac
   ./comp Libtedl
   ./comp Libtemo
   ./comp Libterx
   ./comp Libtfi
   ./comp Libtjo
   ./comp Libtpr
   ./comp Libtsx
   ./comp Tdrive
   ./comp Tinkin
   ./comp Toys
endif
#
unset echo
echo ------------------- done with all compilations --------------------
date
time
-------------- next part --------------
#!/bin/csh
#
#  16 Nov 01 - MWS - C-shell script to compile one MODULE of GAMESS.
#
#  Invoke this script by 'comp MODULE >& MODULE.log &'.
#  Sometimes systems with the Rand MH installed already have a "comp"
#  command as part of the system.  The best fix for this is to put the 
#  current path (.) ahead of all system paths in your PATH variable.
#
#        ----- The next three lines require your attention -----
#  Choose your machine, the legal values for TARGET are
#      compaq-axp,compaq-shmem,convex-spp,cray-pvp,cray-sv1,cray-t3e,
#      fuji-ap,fuji-vpp,hitachi,hp9000,ibm32,ibm64,ibm-sp,linux-pc,
#      macosx,necsx,sgi-mpi,sgi,sun
#  Choose your GAMESS directory tree.
#
#      AHE 3-8-2001: inserting a trial target linux-pc-mpi
#
#
set TARGET=linux-pc-mpi
chdir /home/edwardsa/gamess-current/gamess
#
#    ---- and now the script begins...
#
unset echo
set MODULE=$1
set ACTONLY=$2   # this argument is almost always omitted
echo ======================== $MODULE ==============================
date
#
#   ----- Is this a vector machine? -----
#
set VECTORIZE=false
if ($TARGET == cray-pvp) set VECTORIZE=true
if ($TARGET == cray-sv1) set VECTORIZE=true
if ($TARGET == fuji-vpp) set VECTORIZE=true
if ($TARGET == necsx)    set VECTORIZE=true
#
#   ----- on sequential scalar machines only -----
#   Are level 3 BLAS in a system library?
#   If so, matrix multiplies in MTHLIB.SRC can use DGEMM calls
#
set BLAS3=true
if  ($TARGET == compaq-shmem) set BLAS3=true
if (($TARGET == compaq-axp) && (-e /lib/libdxml.a)) set BLAS3=true
if  ($TARGET == cray-pvp)     set BLAS3=true
if  ($TARGET == cray-sv1)     set BLAS3=true
if  ($TARGET == cray-t3e)     set BLAS3=true
if  ($TARGET == fuji-vpp)     set BLAS3=true
if  ($TARGET == hitachi)      set BLAS3=true
if  ($TARGET == hp9000)       set BLAS3=true
if  ($TARGET == ibm32)        set BLAS3=true
if  ($TARGET == ibm64)        set BLAS3=true
if  ($TARGET == ibm-sp)       set BLAS3=true
if (($TARGET == sun) && (-e /opt/SUNWspro/lib/libsunperf.so)) set BLAS3=true
#
#   ----- delete scalar code before compiling on vector systems -----
#   UNIX's 'ld' loader is incapable of loading two copies of the
#   object code for a routine, and simply using the first one.
#   So, if we are using VECTOR.SRC on this machine, we must split
#   up some of the modules, delete the scalar code for the routines
#   which are found in VECTOR.SRC, and then recombine the rest of
#   the code in that module.  Ugh!
#
if ($VECTORIZE == true) then
   rm -f *.f
   switch ($MODULE)
      case int2a:
         echo Deleting scalar code forms.f
         cp source/int2a.src int2a.src
         fsplit int2a.src
         rm -f int2a.src
         if (-e forms.f) rm forms.f
         if (-e FORMS.f) rm FORMS.f
         cat *.f > int2a.src
         rm -f *.f
         set SRCDIR=.
         breaksw
      case grd2c:
         echo Deleting scalar code dspdfv, jdxyzv, jkgnmv, jkxyzv
         cp source/grd2c.src grd2c.src
         fsplit grd2c.src
         rm -f grd2c.src
         if (-e dspdfv.f) rm -f dspdfv.f jdxyzv.f jkgnmv.f jkxyzv.f
         if (-e DSPDFV.f) rm -f DSPDFV.f JDXYZV.f JKGNMV.f JKXYZV.f
         cat *.f > grd2c.src
         rm -f *.f
         set SRCDIR=.
         breaksw
      case hss2b:
         echo Deleting scalar code jddspd.f
         cp source/hss2b.src hss2b.src
         fsplit hss2b.src
         rm -f hss2b.src
         if (-e jddspd.f) rm -f jddspd.f
         if (-e JDDSPD.f) rm -f JDDSPD.f
         cat *.f > hss2b.src
         rm -f *.f
         set SRCDIR=.
         breaksw
      case eigen:
         echo Deleting scalar code gldiag.f
         cp source/eigen.src eigen.src
         fsplit eigen.src
         rm -f eigen.src
         if (-e gldiag.f) rm -f gldiag.f
         if (-e GLDIAG.f) rm -f GLDIAG.f
         cat *.f > eigen.src
         rm -f *.f
         set SRCDIR=.
         breaksw
      case mthlib:
         echo Deleting scalar code mrarbr, mrtrbr, mtarbr, tfsqb, tftri.
         cp source/mthlib.src mthlib.src
         fsplit mthlib.src
         rm -f mthlib.src
         if (-e mrarbr.f) rm -f mrarbr.f mrtrbr.f mtarbr.f tfsqb.f tftri.f
         if (-e MRARBR.f) rm -f MRARBR.f MRTRBR.f MTARBR.f TFSQB.f TFTRI.f
         cat *.f > mthlib.src
         rm -f *.f
         set SRCDIR=.
         breaksw
      default:
         set SRCDIR=source
         breaksw
   endsw
else
   set SRCDIR=source
   if ($MODULE == blaswrap) set SRCDIR=misc
endif
#
#        optional SIMOMM method using Tinker MM program
set TINKER=false
if ($MODULE == Libtad)  set TINKER=true
if ($MODULE == Libteac) set TINKER=true
if ($MODULE == Libtedl) set TINKER=true
if ($MODULE == Libtemo) set TINKER=true
if ($MODULE == Libterx) set TINKER=true
if ($MODULE == Libtfi)  set TINKER=true
if ($MODULE == Libtjo)  set TINKER=true
if ($MODULE == Libtpr)  set TINKER=true
if ($MODULE == Libtsx)  set TINKER=true
if ($MODULE == Tdrive)  set TINKER=true
if ($MODULE == Tinkin)  set TINKER=true
if ($MODULE == Toys)    set TINKER=true
if ($TINKER == true) then
   set SRCDIR=.
   set OBJDIR=../object
   chdir tinker
   goto cmp
endif
set OBJDIR=object
#
#   ----- Does this module need to be activated? -----
#   Note that if "DOUBLE PRECISION" translates to more than a 64
#   bit floating point quantity, all modules must be passed into
#   the activator to ensure the source is made single precision.
#
if ($TARGET == cray-pvp) goto act
if ($TARGET == cray-sv1) goto act
if ($TARGET == cray-t3e) goto act
if ($TARGET == ibm64)    goto act
if ($TARGET == necsx)    goto act
#
#   Any module with machine dependent code must be activated...
#
if ($MODULE == aldeci) goto act
if ($MODULE == cphf)   goto act
if ($MODULE == cprohf) goto act
if ($MODULE == ddi)    goto act
if ($MODULE == ddishm) goto act
if ($MODULE == dft)    goto act
if ($MODULE == dftint) goto act
if ($MODULE == grd2a)  goto act
if ($MODULE == gugdga) goto act
if ($MODULE == gugdgb) goto act
if ($MODULE == gugdm2) goto act
if ($MODULE == gugem)  goto act
if ($MODULE == gugsrt) goto act
if ($MODULE == gvb)    goto act
if ($MODULE == hss2a)  goto act
if ($MODULE == int2a)  goto act
if ($MODULE == iolib)  goto act
if ($MODULE == lagran) goto act
if ($MODULE == local)  goto act
if ($MODULE == locpol) goto act
if ($MODULE == mccas)  goto act
if ($MODULE == mcqdpt) goto act
if ($MODULE == mcqud)  goto act
if ($MODULE == mcscf)  goto act
if ($MODULE == mctwo)  goto act
if ($MODULE == morokm) goto act
if ($MODULE == mp2)    goto act
if ($MODULE == mp2grd) goto act
if ($MODULE == ordint) goto act
if ($MODULE == rhfuhf) goto act
if ($MODULE == tdhf)   goto act
if ($MODULE == trans)  goto act
if ($MODULE == trfdm2) goto act
if ($MODULE == unport) goto act
if ($MODULE == vector) goto act
#
#   We may need to do a hack below on the matrix multiply routines
#
if (($BLAS3 == true) && ($MODULE == mthlib)) goto act
#
#   On Hitachi, zheev.src should be changed by a hack below
#
if (($MODULE == zheev) && ($TARGET == hitachi)) goto act
#
#   All other modules are straight FORTRAN, just copy them.
#
echo Copying source code, $MODULE.src does not require activation.
cp $SRCDIR/$MODULE.src $MODULE.f
goto cmp
#
#   ----- Choose which version is to be activated -----
#
act:
#
if ($TARGET == convex-spp) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*XXX'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*HP '
endif
if ($TARGET == cray-pvp) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddi)    setenv MACHIN '*SEQ'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*CRY'
   if ($MODULE == vector) setenv MACHIN '*CRY'
endif
if ($TARGET == cray-sv1) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddi)    setenv MACHIN '*MPI'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*CRY'
   if ($MODULE == vector) setenv MACHIN '*CRY'
endif
#       the T3E has a special SHMEM replacement for ddi.src
if ($TARGET == cray-t3e) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddishm) setenv MACHIN '*T3E'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*T3E'
endif
if ($TARGET == compaq-axp) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*DEC'
endif
#       SHMEM on Compaq has a special SHMEM replacement for ddi.src
if ($TARGET == compaq-shmem) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddishm) setenv MACHIN '*DEC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*DEC'
endif
if ($TARGET == fuji-vpp) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*XXX'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*FUJ'
   if ($MODULE == vector) setenv MACHIN '*FUJ'
endif
if ($TARGET == fuji-ap) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*XXX'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*SUN'
endif
if ($TARGET == hitachi) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*XXX'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*HIT'
endif
if ($TARGET == hp9000) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*HP '
endif
if (($TARGET == ibm32) || ($TARGET == ibm64)) then
   if ($TARGET == ibm32)  setenv MACHIN '*I32'
   if ($TARGET == ibm64)  setenv MACHIN '*I64'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*AIX'
endif
if ($TARGET == ibm-sp) then
                          setenv MACHIN '*I32'  # there is no 64 bit MPI lib
   if ($MODULE == ddi)    setenv MACHIN '*MPI'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*AIX'
endif
if ($TARGET == linux-pc) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*F2C'
endif
if ($TARGET == linux-pc-mpi) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*MPI'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*F2C'
endif
if ($TARGET == macosx) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*F2C'
endif
if ($TARGET == necsx) then
                          setenv MACHIN '*I64'
   if ($MODULE == ddi)    setenv MACHIN '*SEQ'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*NEC'
   if ($MODULE == vector) setenv MACHIN '*NEC'
endif
if ($TARGET == sgi-mpi) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*MPI'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*SGI'
endif
if ($TARGET == sgi) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*SGI'
endif
if ($TARGET == sun) then
                          setenv MACHIN '*I32'
   if ($MODULE == ddi)    setenv MACHIN '*SOC'
   if ($MODULE == iolib)  setenv MACHIN '*UNX'
   if ($MODULE == unport) setenv MACHIN '*SUN'
endif
#
#   Here we catch machines for which porting of message passing has
#   not yet been done...
#   Please note that this is not just as simple as picking the
#   string to be activated in DDI above, as you will need to at
#   least think about your compiling clause below.
#
if (($MODULE == ddi) && ("$MACHIN" == "*XXX")) then
   echo "--------------------- error ----------------------------"
   echo The 'comp' script does not know how to compile DDI.SRC.
   echo              Compilation is bombing...
   echo "--------------------- error ----------------------------"
   exit 8
endif
#
#   ----- Activate the source code to produce FORTRAN -----
#
#   On Hitachi, to get the normal Unix calling arguments for -getenv-
#   which is called by ACTVTE, we need a special run time option:
if ($TARGET == hitachi) setenv HF_90OPTS '-FPORT(GETENV)'
#
set echo
setenv   SRCIN $SRCDIR/$MODULE.src
setenv CODEOUT $MODULE.f
#
tools/actvte.x
unset echo
#
#   --------------- special machine "sed hacks" --------------------
#   Not everything can be done with ACTVTE, so we use "sed" (ugh!).
#
#    Direct access record length is measured in bytes on most Unix,
#    but some kinds insist in measuring 4-byte words.  This changes
#    only one line in RAOPEN and one line in OPENDA in IOLIB.SRC.
#    SGI's f90 should not execute the following, but f77 must.
#
if (($MODULE == iolib) && \
   ($TARGET == compaq-axp || $TARGET == compaq-shmem || \
    $TARGET == sgi-mpi || $TARGET == sgi)) then
   mv iolib.f iolib.junk
   sed -e "/     \*      RECL=8\*IRECLN/s//     *      RECL=2*IRECLN/" \
     iolib.junk > iolib.f
   rm -f iolib.junk
endif
#
#   Use DGEMM matrix multiply if possible.
#
if (($MODULE == mthlib) && ($BLAS3 == true)) then
echo blas3 insertion
   mv mthlib.f mthlib.junk
   sed -e "/\*BL3/s//    /" mthlib.junk > mthlib.f
   rm -f mthlib.junk
endif
#
#   Old IBM workstations are 32 bit, but new ones are 64 bit
#
if (($MODULE == unport) && ($TARGET == ibm64)) then
   mv unport.f unport.junk
   sed -e "/      NWDVAR = 2/s//      NWDVAR = 1/" unport.junk > unport.f
   rm -f unport.junk
endif
#
#   This hack is necessary only if compiler flags below request the
#   use of 64 bit integers in GAMESS, and if the vendor's BLAS library
#   insists that its integer arguments be 32 bit quantities.
#   There is a special wrapper file gamess/misc/blaswrap.src that deals
#   with this problem, provided we hack the source to call the wrappers.
#
if (($TARGET == ibm64) && ($MODULE != blaswrap)) then
   sed -e "s/DASUM/XASUM/g"   -e "s/dasum/XASUM/g" \
       -e "s/DAXPY/XAXPY/g"   -e "s/daxpy/XAXPY/g" \
       -e "s/DCOPY/XCOPY/g"   -e "s/dcopy/XCOPY/g" \
       -e "s/DDOT/XDOT/g"     -e "s/ddot/XDOT/g" \
       -e "s/DNRM2/XNRM2/g"   -e "s/dnrm2/XNRM2/g" \
       -e "s/DROT/XROT/g"     -e "s/drot/XROT/g" \
       -e "s/DROTG/XROTG/g"   -e "s/drotg/XROTG/g" \
       -e "s/DSCAL/XSCAL/g"   -e "s/dscal/XSCAL/g" \
       -e "s/DSWAP/XSWAP/g"   -e "s/dswap/XSWAP/g" \
       -e "s/IDAMAX/IXAMAX/g" -e "s/idamax/IXAMAX/g" \
       -e "s/DGEMV/XGEMV/g"   -e "s/dgemv/XGEMV/g" \
       -e "s/DGEMM/XGEMM/g"   -e "s/dgemm/XGEMM/g" \
       $MODULE.f >> $MODULE.munged
   mv $MODULE.munged $MODULE.f
endif
#
#   The following hack selects AOINTS=DIST as the default on high end
#   Compaq and IBM systems, where we know we have a high quality network.
#
if (($MODULE == unport) &&  \
   ($TARGET == compaq-shmem || $TARGET == ibm-sp)) then
   mv unport.f unport.junk
   sed -e "/\*TRF/s//    /" unport.junk > unport.f
   rm -f unport.junk
endif
#
#   Fujitsu AP3000, enable stripping of path information for local I/O files
#
if (($MODULE == iolib) && ($TARGET == fuji-ap)) then
   mv iolib.f iolib.junk
   sed -e "/ISTRIP=1"/s//ISTRIP=0/ \
            iolib.junk > iolib.f
   rm -f iolib.junk
endif
#
#   Hitachi prefers a different double precision complex syntax
#
if (($MODULE == zheev) && ($TARGET == hitachi)) then
   mv zheev.f zheev.junk
   sed -e "/COMPLEX\*16 FUNCTION ZLADIV/s//COMPLEX FUNCTION ZLADIV\*16/" \
       -e "/COMPLEX\*16 FUNCTION ZDOTC/s//COMPLEX FUNCTION ZDOTC\*16/" \
       zheev.junk > zheev.f
   rm -f zheev.junk
endif
#
#   On Hitachi, the first argument of HF_CFUNC is actually a name of C
#   routine, so it is case sensitive.
#
if (($MODULE == unport) && ($TARGET == hitachi)) then
   mv unport.f unport.junk
   sed -e "/HF_CFUNC('ABORT'/s//HF_CFUNC('abort'/" \
       unport.junk > unport.f
   rm -f unport.junk
endif
#
#   On Sun, avoid ugly warning about possible floating point errors issued
#   by every data server process, which come from a FORTRAN STOP statement.
#
if (($MODULE == ddi) && ($TARGET == sun)) then
   mv ddi.f ddi.junk
   sed -e "/      STOP/s//      CALL EXIT(0)/" ddi.junk > ddi.f
   rm -f ddi.junk
endif
#
#   ...and this is the end of the ugly "sed hack"s.
#
#   --------- Now we are ready to compile on the target machine ----------
#
cmp:
#
#   Optionally, save pure FORTRAN in temporary directory for 
#   syntax scan by the nifty FORTRAN analysis program FTNCHEK.
#
if ($ACTONLY == true) then
   mv $MODULE.f ~/scr
   exit
endif
#
if (-e $OBJDIR/$MODULE.o) rm -f $OBJDIR/$MODULE.o
#
#  Options for the Compaq AXP FORTRAN 5.2 compiler under Tru64 V4.0F are
#     -O0 turns off all optimization
#     -O1 does local optimizations
#     -O2 also does global optimizations
#     -O3 also does additional global optimizations
#     -O4 also does routine inlining (default)
#     -O5 also does software pipelining
#     -fast implies less accurate but faster libraries, etc. and -O4.
#     -c means compile only
#     -i8 selects use of 64 bit integers
#     -v gives compiler phase information, and timings
#     -automatic and -static use/avoid stack for local variables
#     -col72 chops source lines past 72 columns
#     -stand nosemantic means don't warn of nonANSI usage (default)
#     -stand nosource_form means don't warn of tabs/lowercase (default)
#     -stand nosyntax means don't warn of nonANSI syntax (default)
#
#  Note that we've tried f77 3.3, 3.7, 3.8, and 5.2, but nothing in between.
#  This is an excellent compiler, and probably all versions work.
#
if ($TARGET == compaq-axp) then
   set OPT = '-O4'
   set echo
   f77 -c -i8 -v -automatic -col72 $OPT $MODULE.f
   unset echo
endif
#      The SuperCluster assumes a specific AXP chip...
if ($TARGET == compaq-shmem) then
   set OPT = '-fast -tune ev67 -arch ev67'
   set echo
   f77 -c -i8 -v -col72 $OPT $MODULE.f
   unset echo
endif
#
#    Cray parallel vector processor (e.g. T90) and SV1 machines
#    Cray T90 tested in February of 2000
#    Cray SV1 rev C with CF90 3.4.0.0.4 tested in March of 2000
#    Note that all Cray clauses now use f90 compiler only.
#    Both classes of machines used identical compiler options.
#
if (($TARGET == cray-pvp) || ($TARGET == cray-sv1)) then
   set OPT = '-O2 '
   if ($MODULE == int2a)  set OPT='-Ovector0,scalar3 '
   if ($MODULE == grd2a)  set OPT='-Ovector0,scalar2 '  # Jerry says
   if ($MODULE == grd2b)  set OPT='-Ovector0,scalar3 '
   if ($MODULE == grd2c)  set OPT='-Ovector0,scalar3 '
   if ($MODULE == gugem)  set OPT='-Ovector0,scalar3 '
   if ($MODULE == vector) set OPT='-O1 '
   if ($MODULE == gvb)    set OPT='-O1 '   # use -O0 if exam15,16,21 fail
   if ($MODULE == gamess) set OPT='-O0 '
   if ($MODULE == efinp)  set OPT='-O0 '
   if ($MODULE == efinta) set OPT='-Ovector0,scalar3 '
   if ($MODULE == efintb) set OPT='-Ovector0,scalar3 '
   if ($MODULE == efgrda) set OPT='-Ovector0,scalar3 '
   if ($MODULE == efgrdb) set OPT='-Ovector0,scalar3 '
   if ($MODULE == efgrdc) set OPT='-Ovector0,scalar3 '
   if ($MODULE == efgrd2) set OPT='-Ovector0,scalar3 '
   set echo
   f90 -c -V $OPT $MODULE.f
   unset echo
endif
#
#  Compiler flags for Cray T3E under UNICOS/mk 2.0.3.18, cf90 3.1.0.3
#  -Oscalarn is various scalar optimizations (n=0,1,2,3)
#  -Oaggress increases table sizes used during optimizations
#  -O[no]msgs,[no]negmsgs [disables]enables compiler optimization messages
#  -dp disables double precision
#
if ($TARGET == cray-t3e) then
   set OPT = '-Oscalar3,aggress'
   if ($MODULE == grd2c) set OPT='-O1'   # for cf90 3.1.0.3 and others
   if ($MODULE == hss1b) set OPT='-O1'   # for cf90 3.4.0.0
   set echo
   env TARGET=cray-t3e f90 -c $OPT -dp -Onomsgs,nonegmsgs $MODULE.f
   unset echo
endif
#
#  Fujitsu VPP300 options
#  Modified by Anthony J. Russell, ANU Supercomputer Facility, 20/1/98
#
#  Should also be suitable for other VPP and VX supercomputers.
#              
#  Options for Fujitsu's UXP/V Fortran90 compiler under UXP/M are
#    -Ab           means reference dummy arguments by address
#    -c            means compile only
#    -Oe           is for extended scalar optimization
#    -Ss           turns off I and W level messages
#    -Kfast        produces code optimized for speed of execution
#                  on the compile time machine
#    Vectorization options
#    -Wv,-Of,-te,-ilfunc    invokes the vector compiler
#                  with full optimization for vector intrinsic functions
#    -te           Expands the work area for vectorization
#    -ilfunc       inlines function calls
#
if ($TARGET == fuji-vpp) then
   set OPT = '-Oe -Kfast -Wv,-Of,-te,-ilfunc'
#         next ones are faster in scalar
   if ($MODULE == int2a)  set OPT='-Oe'
   if ($MODULE == grd2a)  set OPT='-Oe'
   if ($MODULE == grd2b)  set OPT='-Oe'
   if ($MODULE == grd2c)  set OPT='-Oe'
   if ($MODULE == gugem)  set OPT='-Oe'
   if ($MODULE == mctwo)  set OPT='-Oe'
   set INCPATH='-I/usr/lang/mpi/include'
   set echo
   frt -c -Ab -Ss $OPT $MODULE.f $INCPATH
   unset echo
endif
#
#  Fujitsu AP-3000 options
#  this is an array of UltraSPARC chips used to construct a parallel computer
#  Anthony J. Russell, ANU Supercomputer Facility, 5/2/98
#  
#  The OPT string is fastest compiler generated code
#  -c compile only
#  -w66 suppress the messages generated by using F66 features
#  -Nl indicates continuation line
#
if ($TARGET == fuji-ap) then
   set OPT = '-fast -xarch=v8plus'
   set echo
   f77 -c -w66 -Nl30 $OPT $MODULE.f
   unset echo
endif
#
#    Options for HITACH SR2201 machine's HI-UX/MPP compiler
#      blank implies full scalar optimization
#      -W0,'OPT(O(0))' turns off optimization
#      -W0,'OPT(O(ss)),PVEC(DIAG(1))' is vectorization (not tested yet)
#
if ($TARGET == hitachi) then
   set FORT='f90'
   set INCPATH='-I/usr/mpi/include'
   set OPT=" "
   if ($MODULE == sobrt) set OPT="-W0,'OPT(O(0))'"
   set echo
   $FORT -c $OPT $INCPATH $MODULE.f
   unset echo
endif
#
#  HP-UX B.11.0, using f90 2.4.10 (f90 +version reports version number)
#  The options mean:
#    +[no]save governs automatic variable allocation strategy
#    +On   selects optimization level; n=0,1,2,3,4  (-O means +O2)
#          0 is none, 1 is block transforms, 2 is full within subprograms
#          3 crosses subprograms (includes inlining) also loop transforms
#    +Oaggressive selects the following four additions to +O2:
#    +Oentrysched  = instruction scheduling on entry/exit
#    +Olibcalls    = special trig and power functions (lacking error hand.)
#    +Onofltacc    = permits use of a faster single multiply/add instruction.
#    +Onoinitcheck = disable initialization of any unitialized datum
#       other optimizations include
#    +Onolimit     = ignores time and memory limits when optimizing
#    +Ofastaccess  = gives fast access to global data
#    +U77 gives access to BSD's 3F library
#
if ($TARGET == hp9000) then
                          set OPT='+O2 +Oaggressive +Onolimit +Ofastaccess'
   if ($MODULE == unport) set OPT='+O2 +Oaggressive +Onolimit +Ofastaccess +U77'
   set echo
   f90 -c $OPT +nosave $MODULE.f
   unset echo
endif
#
#  The following was used for compiler version 10.20 under HP-UX 10.
#
#---if ($TARGET == hp9000) then
#---   set OPT='+O2'
#---   if($MODULE == hss1a)  set OPT='+O1'
#---   set echo
#---   f77 -c $OPT +U77 +Oentrysched +Ofastaccess \
#---           +Olibcalls +Onofltacc +Onolimit $MODULE.f
#---   unset echo
#---endif
#
if ($TARGET == convex-spp) then
   set OPT='+O2 +Obb600'
   if ($TARGET == convex-spp) set OPT='$OPT -K'
   if ($MODULE == guess)  set OPT='+O1'
   if ($MODULE == hss2a)  set OPT='+O1'
   if ($MODULE == inputa) set OPT='+O1'
   set echo
   f77 -c $OPT +FPD +E1 +E6 +T -s $MODULE.f
   unset echo
endif
#
#   For IBM 32 or 64 bit RS/6000 workstations or SP parallel systems,
#   the options for XL FORTRAN 7.1.0 and AIX 4.3.3 are
#       -c means compile only
#       -O2 and -O3 choose optimization levels (there is no -O1)
#       -qarch=com,601,pwr,pwr2,pwr3 (see also -qtune)
#       -qflag requests I,L,W,E,S,Q (information, language, 
#              warning, error, severe, quiet) messages
#       -qflttrap=ov:zero:inv:inex:enable traps bad arithmetic
#       -qhalt stops compiler after certain error level
#       -qnosave forces non-static storage
#       -qinitauto=ff initializes non-static storage
#       -qextchk checks calling args and common lengths at link
#       -qsource requests a printable .lst file.
#       -qtune selects target chip
#
#   The following clause requires XL FORTRAN 3.2 or higher.
#   Use of 64 bit integers requires XL FORTRAN 6.1 or higher.
#   The command to find your FORTRAN version level depends on your
#   AIX version: v3 is "lslpp -h xlfcmp.obj", v4="lslpp -h xlfcmp".
#
#   To profile code during development stages,
#   1. "comp" desired modules with -pg added to the xlf flags
#   2. "lked" after adding -p to the xlf flags.
#   3. execute to completion, generating 'mon.out' file.
#   4. profile by 'prof gamess.00.x mon.out'.
#
if (($TARGET == ibm32) || ($TARGET == ibm64) || ($TARGET == ibm-sp)) then
   set FORT='xlf'
   if ($TARGET == ibm-sp) set FORT='mpxlf'
#         old 32 bit workstations, whether Power, Power2, or PowerPC
#         should all select plain vanilla common architecture.  Its
#         faster and runs on any combination of these chips you own.
#         under xlf 4.1.0 a few files won't fully optimize
   if ($TARGET == ibm32) then
      set  OPT='-O3'
      if ($MODULE == grd2b)  set OPT='-O3 -qspillsize=1500'
      if ($MODULE == gugdm)  set OPT='-O2'
      if ($MODULE == gugem)  set OPT='-O2'
      if ($MODULE == resc)   set OPT='-O2'
      if ($MODULE == trnstn) set OPT='-O2'
      set ARCH='com'
      set TUNE='604'   # PowerPC tuning makes even Power1 run faster
   endif
#         new 64 bit workstations are sure to be Power3 chips.
#         everything optimizes under xlf 5.1.1 and 6.1.0 and 7.1.0
   if ($TARGET == ibm64) then
      set  OPT='-O3 -q64 -qintsize=8 -qspillsize=1500'
      set ARCH='pwr3'
      set TUNE='pwr3'
   endif
#         your SP might not be a new one, if not, mimic ibm32 clause.
#         note that MPI-1 on the SP precludes 64 bit compilation.
#         everything optimizes under xlf 5.1.1 and 6.1.0
   if ($TARGET == ibm-sp) then
      set  OPT='-O3 -qspillsize=1500'
      set ARCH='pwr3'
      set TUNE='pwr3'
   endif
   set echo
   $FORT -c $OPT -qarch=$ARCH -qtune=$TUNE \
         -qflag=I:I -qhalt=W -qnosave $MODULE.f
   unset echo
endif
#
#       Our Beowulf is currently using Dead Rat 6.2 so we will use g77 compilation
#
if ($TARGET == linux-pc-mpi) then
   set OPT = '-O2'
   set INCPATH='-I/usr/include/mpi-beowulf/'
   set echo
   g77-3.2 -c $OPT -malign-double -fautomatic \
           -Wno-globals -fno-globals $MODULE.f $INCPATH
   unset echo
endif
#
#   ------ RedHat Linux on "PC" (Pentium/Athlon chips)  -----
#   There are no less than four compilers for Linux included here!
#   They are g77 (default), f2c/gcc, Intel's ifc, and Portland's pgf77.
#   If you want to use one of the other three, just make sure that the
#   one you prefer is uncommented.  In addition, you will need to edit
#   the link-edit script 'lked' to reflect your compiler choice here.
#
#   Note that we have only used Redhat, so this is not necessarily
#   correct for Slackware, SUSE, FreeBSD, ..., although many people
#   have been able to get these others to work.
#
#   1. Using g77 compiler.     compiler version query: rpm -q gcc-g77 
#   If your RedHat is older than 6.0, see the f2c/gcc clause below.
#
#   There is an I/O library problem in g77, apparently in all RedHat
#   releases from 6.0 through 7.1's gcc-g77-2.96-81.  You can find a
#   a fix on our web page: www.msg.ameslab.gov/GAMESS/GAMESS.html,
#   then check the bottom of the "how to get a copy of GAMESS" page.
#   It is crucial to install this, as it works around a very serious
#   error in the g77 I/O library.
#   The 3.0 release of GNU compilers is scheduled to fix this bug.
#
#   The meaning of g77's flags are
#      -O0, -O1, -O2, -O3 are the optimization levels, -O means -O1
#      -Wno-globals -fno-globals together suppress argument type checking.
#      -fautomatic defeats use of static storage
#      -malign-double uses 64 bit data alignment
#   Tests on a representative GAMESS test suite (executing energy, gradient
#   hessian code, as well as various SCF, MCSCF, MP2 runs, show that each
#   of the more aggressive optimizations -ffast-math, -march=i586, and -O3
#   gave speedups of less than 1% each.  (-march=i686 slows a Celeron type
#   Pentium II fractionally).  None of these optimizations are used below.
#      
if ($TARGET == linux-pc) then
   set OPT = '-O2'
   set echo
   g77-3.2 -c $OPT -malign-double -fautomatic \
           -Wno-globals -fno-globals $MODULE.f
   unset echo
endif
#
#   2. RedHat 4.x and 5.x's FORTRAN compiler was f2c/gcc, and early
#   releases of g77 that were present in some of these distributions
#   do not run GAMESS correctly.  If you have RedHat 6.0 or higher,
#   which do not contain the f2c interpreter, see just above.  If
#   your RedHat is older, use f2c/gcc in preference to g77.
#   A corresponding change is required in "lked" if you use f2c/gcc.
#
#--if ($TARGET == linux-pc) then
#--   if (($MODULE == qfmm) || ($MODULE == solib) || ($MODULE == zheev)) then
#--       mv $MODULE.f $MODULE.junk
#--       sed -e s/DREAL/DBLE/g $MODULE.junk > $MODULE.f
#--       rm -f $MODULE.junk
#--   endif
#--   set OPT = '-O3 -malign-double'
#--   set echo
#--   f2c -w66 -a -Nn802 -Nx400 $MODULE.f
#--   gcc -c $OPT $MODULE.c
#--   rm -f $MODULE.c
#--   unset echo
#--endif
#
#    3. Intel Fortran Compiler 5.0.1 for Linux
#    This compiler can be downloaded for non-commercial purposes:
# http://developer.intel.com/software/products/compilers/f50/linux/noncom.htm
#    -O3  is the most aggressive optimization, the default is -O2.
#    -cm  turn off printing of informational messages (f90 syntax, etc)
#    -w   turn off all warnings
#    Remember to load compiler setup by running proper (for your shell)
#    setup script as described in Intel's docs:
#            . /opt/intel/compiler50/ia32/bin/ifcvars.sh
#
#--if ($TARGET == linux-pc) then
#--   set OPT = '-O3'
#--   if ($MODULE == zheev) set OPT=' '
#--   set echo
#--   ifc $OPT -cm -w -c $MODULE.f
#--   unset echo
#--endif
#
#    4. Portland Group compiler.  
#    It is recommended that you use the native g77 compiler clause above.
#    This is a synthesis of recommendations from Fred Arnold and Brian
#    Salter-Duke.  -fast means -O2 -Munroll -Mnoframe, the latter option
#    seems to be the source of several incorrect results.  It is said that
#    "-Munroll" and "-tp p6" gives no significant improvement in run times.
#
#--if ($TARGET == linux-pc) then
#--   setenv PATH $PATH\:/usr/pgi/linux86/bin
#--   setenv PGI /usr/pgi
#--   if (($MODULE == qfmm) || ($MODULE == solib) || ($MODULE == zheev)) then
#--       mv $MODULE.f $MODULE.junk
#--       sed -e s/DREAL/DBLE/g $MODULE.junk > $MODULE.f
#--       rm -f $MODULE.junk
#--   endif
#--   set OPT = '-O2'
#--   set echo
#--   pgf77 -c $OPT $MODULE.f
#--   unset echo
#--endif
#
#  MacOS X (aka Darwin)
#  Note you must have both the Apple Developer Tools installed and F2C
#  installed before attempting to compile GAMESS.
#
if ($TARGET == macosx) then
    if (($MODULE == qfmm) || ($MODULE == solib) || ($MODULE == zheev)) then
        mv $MODULE.f $MODULE.junk
        sed -e s/DREAL/DBLE/g $MODULE.junk > $MODULE.f
        rm -f $MODULE.junk
    endif
    set OPT = '-O3'
    set echo
    f2c -w66 -a -Nn802 -Nx400 $MODULE.f
    cc -c $OPT $MODULE.c
    rm -f $MODULE.c
    unset echo
endif
#
# Options for f90 under SUPER-UX on the NEC SX Series are
#      -float0 -w Cray compatible data
#      -b sets needed passing of scalar arguments
#      -pvtcl noassume vwork=stack sets vectorization temporaries parameters
#      -Nv  vectorization is turned off
#
if ($TARGET == necsx) then
   set OPT = '-Wf"-v -pvctl noassume vwork=stack"'
   if ($MODULE == mcqdpt) \
       set OPT='-Wf"-v -pvctl noassume vwork=stack noaltcode nodivloop -NO"'
   if ($MODULE == ddi) \
       set OPT='-Wf"-v -pvctl noassume vwork=stack -I /usr/include"'
#         compiler character problem
   if ($MODULE == inputc) set OPT='-Wf"-Nv"'
#         next ones are faster in scalar
   if ($MODULE == int2a)  set OPT='-Wf"-Nv"'
   if ($MODULE == int2b)  set OPT='-Wf"-Nv"'
   if ($MODULE == grd2a)  set OPT='-Wf"-Nv"'
   if ($MODULE == grd2b)  set OPT='-Wf"-Nv"'
   if ($MODULE == grd2c)  set OPT='-Wf"-Nv"'
   if ($MODULE == gugem)  set OPT='-Wf"-Nv"'
   if ($MODULE == gugdrt) set OPT='-Wf"-Nv"'
#
   set echo
   f90 -c -float0 -ebw $OPT $MODULE.f
   unset echo
endif
#
#  SGI machines with MPI library installed.  For example, a Origin 2000.
#
#  We recommend that you do not try for optimization level -O3.
#  ARCH might need to be -r8000 or -r5000, type "hinv -c processor" to see.
#  The options "-64 -mips4" might need to be "-n32 -mips3", perhaps, be
#  sure to make the corresponding change in compall and lked scripts too.
#
#  If you use f90, do not reset the units for RECL in the sed hacks above.
#
if ($TARGET == sgi-mpi) then              
   set OPT='-O2'
   if ($MODULE == inputc) set OPT=' '
   set ARCH='-r10000'
   set FLAGS='-64 -mips4 -automatic -G0 -OPT:IEEE_arith=3:roundoff=3:Olimit=0'
   set echo                    
   f77 -c -show $OPT $ARCH $FLAGS $MODULE.f
   unset echo                                          
endif
#
#  SGI machines without MPI, using sockets.   tested for Irix 5.1 with f77 4.0
#
#  This is for a very old Indy R4x000 system, as it uses the the -mips2 flag.
#  If you have a system in between an Origin 2000 and an Indy, you might
#  wish to use control language indentical to that above here.  If so, you
#  must also change the compall and lked scripts to use "-64 -mips4" flags
#  rather than "-mips2", but otherwise rely on sockets to implement DDI.
#
#  If you use f90, do not reset the units for RECL in the sed hacks above.
#
if ($TARGET == sgi) then
   set OPT = '-O2'
   if ($MODULE == cprohf) set OPT = '-O1'
   if ($MODULE == inputc) set OPT = '-O1'
   set echo
   set FLAGS='-mips2 -v -automatic -G 0 -Olimit 2000 -Nl300'
   f77 -c $OPT $FLAGS $MODULE.f
   unset echo
endif
#
#  Options for UltraSPARC Sun f77 4.2 under Solaris 2.6.
#     -c compile only
#     -v print verbose information on compiler phases and options
#     -stackvar forces non-static storage
#     -fast     is a smorgasbord of all the below, if you wish to
#               override one you must *follow* the -fast with it.
#         -xtarget=native  (optimize for chip being compiled upon)
#         -O4        (see below)
#         -libmil    (inline some math routines)
#         -fsimple=1 (0,1,2 control floating point simplifications)
#         -dalign    (double word alignment and instructions)
#         -xlibmopt  (link to optimized math library)
#         -depend    (DO loop data dependency analysis)
#         -fns       (fast underflow handling)
#         -ftrap=%none  (floating point error handling)
#     -O5 = aggressive optimizations
#     -O4 = automatic inlining
#     -O3 = loop unrolling and additional global optimizations
#     -O2 = basic local, global optimizations (must turn off -depend)
#     -O1 = minimal optimization
#     -xtarget=generic doesn't tune for any particular chip
#     -xtarget=native selects a -xarch,-xchip,-xcache for compile host
#            you may want to set this a particular chip if you run
#            more times on a different system than you compile on.
#  On an UltraSPARC, the -xtarget=native in -fast implied -xarch=v8!
#  You get best results by picking -xarch value closest to your system.
#     -xarch=generic  no particular SPARC chip
#     -xarch=v7      SPARC V7   (i.e. SPARCstn 1)
#     -xarch=v8      SuperSPARC (i.e. SPARCstn 2 and SPARCstn 10)
#     -xarch=v8a     microSPARC
#     -xarch=v8plus  UltraSPARC II, using 32 bit instructions
#     -xarch=v8plusb UltraSPARC III
#     -xarch=v9      64 bit instruction set (not tested well)
#
#  MPI usage: in comp, add -I/opt/SUNWhpc/include/
#             in lked, add -L/opt/SUNWhpc/lib -lfmpi -lmpi
#  as well as making the obvious changes to move Sun clauses from the
#  TCP/IP sockets sections to the MPI sections in compall and lked.
#
#  If you have a pre-ultraSPARC system, try OPT='-O2' or '-O3' while
#  eliminating the -xarch flag, and you might also need -Nl30.
#
if ($TARGET == sun) then
   set OPT='-fast -O4 -xarch=v8plus'
   if ($MODULE == ddi)    set OPT='-fast -O4 -nodepend -xarch=v8plus'
   if ($MODULE == grd2c)  set OPT='-fast -O4 -nodepend -xarch=v8plus'
   set echo
   f77 -c $OPT -stackvar $MODULE.f
   unset echo
endif
#
#  Store the generated object code, clean up, and quit
#
mv $MODULE.o $OBJDIR/$MODULE.o
if ($TINKER == true) exit
rm -f $MODULE.f
if ($SRCDIR == .) rm -f $MODULE.src    # delete copy with no scalar code
unset echo
date
time
-------------- next part --------------
#!/bin/csh
#
#   9 Nov 01 - MWS - C-shell script to link-edit GAMESS, with loadmap.
#
#  Invoke this job by 'lked EXE VERNO >& lked.log &'
#    EXE is normally 'gamess',
#    VERNO is the next version number, e.g. 01, 02, 03, ...
#    This produces executable "EXE.VERNO.x", e.g. gamess.01.x
#
#  ------ The next three lines require your attention.
#  Choose your machine, the legal values for TARGET are
#       compaq-axp,compaq-shmem,convex-spp,cray-pvp,cray-sv1,cray-t3e,
#       fuji-vpp,fuji-ap,hitachi,hp9000,ibm32,ibm64,ibm-sp,linux-pc,
#       macosx,necsx,sgi-mpi,sgi,sun
#  Choose your GAMESS directory tree.
#
set TARGET=linux-pc-mpi
chdir /home/edwardsa/gamess-current/gamess
#
#    ---- and now the script begins...
#
set EXE=$1
set VERNO=$2
if (null$EXE == null) set EXE=gamess
if (null$VERNO == null) set VERNO=00
date
#
echo GAMESS will be linked into the executable image $EXE.$VERNO.x.
#
if (-e $EXE.$VERNO.x) then
   echo An old executable $EXE.$VERNO.x already exists!
   echo Bombing out...
   exit
endif
#
#   Choose the message passing code, and possible system libraries.
#
set MSG_LIBRARIES=' '
#
switch ($TARGET)
#               SHMEM interface
   case compaq-shmem:
   case cray-t3e:
      set MSG_CODE='ddishm.o'
      breaksw
#               TCP/IP sockets interface
   case compaq-axp:
   case hp9000:
   case ibm32:
   case ibm64:
   case linux-pc:
   case macosx:
   case sgi:
   case sun:
      set MSG_CODE='ddi.o ddisoc.o'
      if ($TARGET == sun) set MSG_LIBRARIES='-lsocket -lnsl'
      breaksw
#               MPI-1 interface
   case cray-sv1:
      set MSG_CODE='ddi.o'
      breaksw
   case ibm-sp:
      set MSG_CODE='ddi.o'
      breaksw
   case sgi-mpi:
      set MSG_CODE='ddi.o'
      set MSG_LIBRARIES='-lmpi -lsma'
      breaksw
   case linux-pc-mpi:
      set MSG_CODE='ddi.o'
      set  MSG_LIBRARIES='/usr/lib/libmpif.a /usr/lib/libbproc.a /usr/lib/libbeostat.a'
      breaksw
#               Running sequentially...
   case cray-pvp:
      set MSG_CODE='ddi.o'
      set MSG_LIBRARIES=' '  # change to "-lmpi -lsma" if you try for MPI-1
      breaksw
   case necsx:
      set MSG_CODE='ddi.o'
      set MSG_LIBRARIES=' '  # change to "/usr/lib0/libmpiw.a" for MPI-1
      breaksw
#               no interface currently available
#       We keep our old notes about what the MPI library is called in
#       case these machines decide to use MPI as the message transport.
#       As these get fixed they should move to the MPI-1 or Sockets
#       sections just above.  MSG_CODE is probably to be ddi.o
#       All of these run on into an abort with error message.
   case convex-spp:
      set MSG_CODE=' '
      set MSG_LIBRARIES=' '
   case fuji-ap:
      set MSG_CODE=' '
      set MSG_LIBRARIES='-Bdynamic -L/opt/FSUNmpiap/lib -lmpi \
                         -L/opt/FSUNaprun/lib -lemi -lmpl'
   case fuji-vpp:
      set MSG_CODE=' '
      set MSG_LIBRARIES='-Wl,-P -L/usr/lang/mpi/lib -lmpi -lmp'
   case hitachi:
      set MSG_CODE=' '
      set MSG_LIBRARIES='-lfmpi -lmpi'

      echo This machine does not have modernized message passing selected yet.
      exit 8
      breaksw
#               and this must be a typo
   default:
      echo Unrecognized machine target
      breaksw
endsw
#
#   optional SIMOMM method using Tinker MM code
#   qmmm.o is dummy substitute from the standard GAMESS distribution.
set TINKER=false
if ($TINKER == true) then
   set QMMMOBJ="Libtad.o Libteac.o Libtedl.o Libtemo.o Libterx.o Libtfi.o"
   set QMMMOBJ="$QMMMOBJ Libtjo.o Libtpr.o Libtsx.o Tdrive.o Tinkin.o Toys.o"
else
   set QMMMOBJ='qmmm.o'
endif
#
#   Define the options for the system loader 'ld'.  These are
#       LDR       - the name of the loader (usually a compiler).
#       LDOPTS    - options, such as asking for a load map.
#       LIBRARIES - extra system libraries which must be searched.
#       BLAS      - set to blas.o if not using system BLAS.
#                   set to blank if using system BLAS.
#       VECTOR    - set to vector.o on a vector processor.
#                   set to blank on a scalar processor.
#       QUICHE    - set to zunix.o if using true dynamic memory,
#                   possibly add other system specific objects.
#                   set to blank otherwise.
#
if ($TARGET == compaq-axp) then
   set LDR='f77'
   set LDOPTS='-Wl,-M'   # add -non_shared to see libraries
   if (-e /lib/libdxml.a) then
      set LIBRARIES='-ldxml'
      set BLAS=' '
   else
      set LIBRARIES=' '
      set BLAS='blas.o'
   endif
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ( $TARGET == compaq-shmem) then
   set LDR='f77'
   set LDOPTS=' '   # add -non_shared to see libraries
   set LIBRARIES='-ldxml -lshmem -lelan'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == convex-spp) then
   set LDR='/usr/convex/bin/fc'
   set LDOPTS='-Wl,+FPD,-a,archive'
   set LIBRARIES='/usr/lib/libveclib.a'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if (($TARGET == cray-pvp) || ($TARGET == cray-sv1)) then
   set LDR='f90'
   set LDOPTS='-m 0 -v -V'
   set LIBRARIES=' '
   set BLAS=' '
   set VECTOR='vector.o'
   set QUICHE='zunix.o'
endif
#
if ($TARGET == cray-t3e) then
   set LDR='env TARGET=cray-t3e cld -Dstreams=on'
   set LDOPTS='-V -Xm'
   set LIBRARIES='-lc -lm -lu -lf -lsci -lfi -lsma -lmpi -lpmpi'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == fuji-vpp) then
   set LDR='frt'
   set LDOPTS='-J'
   set LIBRARIES='-L/opt/blas/blas_vpp300/lib -lblasvpp'
   set BLAS=' ' 
   set VECTOR='vector.o'
   set QUICHE='zunix.o'
endif
#
if ($TARGET == fuji-ap) then
   set LDR='f77'
   set LDOPTS=' '
   set LIBRARIES=' '
   set BLAS='blas.o'
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == hitachi) then
   set LDR='f90'
   set LDOPTS='-L/usr/mpi/lib/hmpp2/cml/'
   set LIBRARIES='-lblas'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == hp9000) then
   set LDR='f90'
   set LDOPTS='+O2 +Ofastaccess +U77 -Wl,-ldld -Wl,-m'
   set LIBRARIES='/opt/fortran90/lib/pa2.0/libblas.a'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#           for f77 under HP-UX version 10
#--if ($TARGET == hp9000) then
#--   set LDR='f77'
#--   set LDOPTS='+O2 +DA2.0 +FPD +U77 +Ofastaccess -Wl,-a,archive'
#--   set LIBRARIES='/opt/fortran/lib/pa1.1/libblas.a'
#--   set BLAS=' '
#--   set VECTOR=' '
#--   set QUICHE='zunix.o'
#--endif
#
#   To see length of common blocks and subroutine sizes,
#     add "-Wl,-bmap:../lked.sizes" to LDOPTS.
#   To avoid use of dynamic link libraries, if xlfrte is absent,
#     add "-Wl,-bnso -Wl,-bI:/lib/syscalls.exp" to LDOPTS.
#   BLAS routines in ESSL are faster than the free lib in all AIX versions,
#   if you have installed ESSL.  Note Power2 has a different ESSL name.
#   The -lblas library has no 64 bit obects in it, and the  -lessl
#   library assumes 32 bit calling args are passed to it.  Therefore
#   there is no usable system library if using 64 bit mode on Power3.
#
if (($TARGET == ibm32) || ($TARGET == ibm64) || ($TARGET == ibm-sp)) then
   set LDR='xlf'
   if ($TARGET == ibm-sp) set LDR='mpxlf'
   set LDOPTS='-Wl,-m  -Wl,-bmaxdata:0x70000000  -Wl,-bloadmap:../lked.map'
   if ($TARGET == ibm64) set LDOPTS='-q64 -Wl,-m -Wl,-bloadmap:../lked.map'
   set LIBRARIES='-lblas'                             # free in every AIX
   if (-e /lib/libessl.a)   set LIBRARIES='-lessl'    # for Pwr, PPC, Pwr3
#--if (-e /lib/libesslp2.a) set LIBRARIES='-lesslp2'  # for Pwr2, P2SC
   set BLAS=' '
   if ($TARGET == ibm64) set BLAS='blaswrap.o'
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
#  Some hand edits will be necessary for Linux,
#  depending on which compiler you used in the 'comp' script:
#             g77                   f2c/gcc      ifc        pgf77
#  LDR        'g77'                 'gcc'        'ifc'      'pgf77'
#  LDOPTS     '-Wl,-M'              ' '          ' '        '-g77libs'
#  LIBRARIES  ' '                   '-lf2c -lm'  '-Vaxlib'  ' '
#  QUICHE     'zunix.o endfile.o'   'zunix.o'    'zunix.o'  'zunix.o'
#
#  Got an unresolved external reference on endfile.o with g77?  It
#  means you didn't compile the I/O library patch mentioned in the 
#  'comp' script between doing the compall and lked steps.  If you
#  are using a 3.0 GNU g77 release, don't use endfile.o in QUICHE.
#
if ($TARGET == linux-pc) then
   set LDR='g77-3.2'
   set LDOPTS='-Wl,-M'
   set LIBRARIES='-L/home/edwardsa/ATLAS/lib/Linux_ATHLON -llapack -lf77blas -lcblas -latlas '
   set BLAS='blas.o'
   set VECTOR=' '
   set QUICHE='zunix.o endfile.o'
endif
if ($TARGET == linux-pc-mpi) then
   set LDR='g77-3.2'
   set LDOPTS=' '    # to see load map: -Wl,-M
   set LIBRARIES=' -L/home/edwardsa/ATLAS/lib/Linux_ATHLON -llapack -lf77blas -lcblas -latlas'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o endfile.o'
endif
#
if ($TARGET == macosx) then
    set LDR='cc'
    set LDOPTS='-Wl,-M'
    set LIBRARIES='-lf2c'
    set BLAS='blas.o'
    set VECTOR=' '
    set QUICHE='zunix.o'
endif
#
if ($TARGET == necsx) then
   set LDR='f90'
#  If MathKeisan is installed do not use the old memory layout.
   if ({(/bin/nm /usr/lib/libblas.a | grep -c mkversion)}) then
      echo 'MathKeisan is in use'
      set LDOPTS='-float0 -w -g -Pmulti '
   else
      echo 'MathKeisan is NOT in use'
      set LDOPTS='-float0 -w -g -Pmulti -Wl,-h32G_memlayout'
   endif
   set LIBRARIES='/usr/lib0/libblas_64.a'
   set BLAS=' '
   set VECTOR='vector.o'
   set QUICHE='zunix.o'
endif
#
if ($TARGET == sgi-mpi) then
   set LDR='f77'
   set LDOPTS='-Wl,-M -64 -mips4'
   set LIBRARIES='-lblas'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == sgi) then
   set LDR='f77'
   set LDOPTS='-Wl,-M -mips2'
   set LIBRARIES='-lblas -lsun'
   set BLAS=' '
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
if ($TARGET == sun) then
   set LDR='f77'
   set LDOPTS='-m'
   if (-e /opt/SUNWspro/lib/libsunperf.so) then
      set LIBRARIES='-xlic_lib=sunperf'
      set BLAS=' '
   else
      set LIBRARIES=' '
      set BLAS='blas.o'
   endif
   set VECTOR=' '
   set QUICHE='zunix.o'
endif
#
set echo
chdir object
$LDR  -o ../$EXE.$VERNO.x   $LDOPTS    \
      gamess.o unport.o $BLAS    $VECTOR  $QUICHE  $MSG_CODE \
      aldeci.o algnci.o \
      basecp.o basext.o bashuz.o bashz2.o basn21.o basn31.o bassto.o  \
      cosmo.o  chgpen.o   cphf.o   cprohf.o delocl.o  \
      dft.o    dftaux.o dftexc.o dftfun.o dftgrd.o dftint.o  \
      dmulti.o drc.o    ecp.o    ecpder.o ecphw.o  ecplib.o ecpsbk.o  \
      efdrvr.o efgrda.o efgrdb.o efgrdc.o efgrd2.o efinp.o   \
      efinta.o efintb.o efelec.o efpaul.o efpcm.o  efpcov.o  \
      eigen.o  ffield.o frfmt.o  globop.o gradex.o guess.o  \
      grd1.o   grd2a.o  grd2b.o  grd2c.o   \
      gugdga.o gugdgb.o gugdm.o  gugdm2.o gugdrt.o gugem.o  gugsrt.o  \
      gvb.o    hess.o   hss1a.o  hss1b.o  hss2a.o  hss2b.o            \
      inputa.o inputb.o inputc.o int1.o   int2a.o  int2b.o  iolib.o   \
      lagran.o local.o  loccd.o  locpol.o  \
      mccas.o  mcjac.o  mcqdpt.o mcqdwt.o mcqud.o  mcscf.o  mctwo.o  \
      morokm.o mp2.o    mp2ddi.o mp2grd.o  \
      mpcdat.o mpcgrd.o mpcint.o mpcmol.o mpcmsc.o mthlib.o  \
      nameio.o ordint.o parley.o  \
      pcm.o    pcmcav.o pcmder.o pcmdis.o pcmief.o pcmpol.o pcmvch.o  \
      prpel.o  prplib.o prppop.o qfmm.o   qmfm.o    \
      raman.o  resc.o   rhfuhf.o rxncrd.o ryspol.o  \
      scflib.o scfmi.o  scrf.o   sobrt.o  soffac.o solib.o  sozeff.o  \
      statpt.o surf.o   symorb.o symslc.o  \
      tdhf.o   trans.o  trfdm2.o trnstn.o trudge.o  \
      vibanl.o vscf.o   zheev.o  zmatrx.o  \
      $QMMMOBJ \
      $MSG_LIBRARIES $LIBRARIES
#
unset echo
time


More information about the Beowulf mailing list