[Beowulf] perl with OpenMPI gotcha?
David Mathog
mathog at caltech.edu
Mon Nov 23 19:32:31 UTC 2020
Sounds like MPI isn't supported very well now, and perhaps it never was.
In each application the remote jobs all run the same script but they
receive slightly different parameters which are read from the
command line. That was easy with PVM since each remote
job start had its own parameter list. Using pdsh those parameters would,
it seems, have to go in through node/job specific configuration files
which the scripts would need to be modified to read. (Or possibly through
environmental variables, but it would still need more modification of the
compute node scripts.) So probably the quickest port will be
to just use Parallel::Forkmanager and in the child processes run
ssh jobs using the existing script parameter lists. That way there will
still be pids to work with and the master can still loop to keep track of
which ones are running or have completed. The jobs operate only on files
on an NFS file system (for input, output, and status) and compute node
local (scratch files, if any). No data is fed in through stdin or returned
through stdout.
Thanks for your help,
David Mathog
More information about the Beowulf
mailing list